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(54) Search/sort circuit for neural networks 



(57) In a neural network comprised of a plurality of 
N neuron circuits, each having calculated the distance 
(D1, .... D4) coded on p bits (D1 =d11 ... d41) between 
an input vector and the prototype vector stored in the 
weight memory thereof, there is disclosed an aggregate 
circuit (51 7) comprised of N search/sort circuits (502-1 , 
.... 502-4) each being placed in a neuron circuit. The 
search/sort circuit is adapted to determine the minimum 
distance among said calculated distances. Each 
search/sort circuit (502-1) is comprised of p elementary 
base units (510-1 1 to 510-41) connected in series and 
disposed in a column direction. The distance bit signals 
of the same bit rank are applied to said base units 
according a line direction. As a consequence, the base 
units of the aggregate circuit are organized in a matrix. 
The feedback signal corresponds to the signal obtained 
by performing an OR function in an OR gate (12.1) 
between all the local output signals generated by the 
base units of a determined line. The search process is 
based on the search of zeroes in the distance bit signals, 
from the MSB's to the LSB's. If a zero is found in a deter- 
mined line, all the columns which have a one in this line 
are excluded from the following search. The process is 
continued until it remains only one distance: the 
searched minimum distance, that is finally available at 
the output of the said OR circuit. The above described 
search/sort circuit can be significantly improved by 
adjoining a latch based circuit so that the aggregate cir- 
cuit is now capable to sort the remaining distances in an 
increasing order. 
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. fillnadate (attorney 

innovative neuron circuit architectures, application number . 9 

n"ars:~uit for serial connection of neuron circuits, application number 
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Sfsi^' ^S^ar^^erg-ehS'S^Sa^^^^^ 
^•"rtSntt^tlSates^ecapitalle.,.^^^^^ 

'^r:rc;irgres;^^^^^^^^^^ 

:^o?.»';.res;™^■e^^-.^.:«=^^ 
rr.rg,^r;^ry-i.c= 

and F2 = 9 instead of seventeen, but st.ll with the ^^"^^ ^^tego^^^^^^ ^ .^^ e^en more, should these letters be 
and the lower case letter "e" are represented by two different category C. Thus, it can be 

i^agrth^'^^I^S^^^^^^^^^^ 

rsra:;h°e%:r~^^^^^^^ 

instance C = 5). input vector F is then f by point P with its two components P1 and 

NOW turning to Fig. 2(A), the stored prototype ^f^f ' L !oTtSSuS(or characteristic) space. Encompassing 
P2 in a two dimension space. This space .s usually refer ed to «^ the feature (o c ^ ^ discriminating 

point P illustrative of prototype vector P. one '"^V f «^?f '"^.^^""^^^^^^^^ and any input (or incoming) 

criteria by the system to appreciate the degree ^^^^''^'j'^^^^J^^^^^^^ during the recognition 

vector (or pattern) representing the character under scan that 

phase. The system will then perform a comparison between the .nput vector A an^^^^^ s w x ^ ^^^^^ 

to appreciate their degree of similitude. This degree of s.m,l.tude J'^y J^^Ts aVsume an input vector A comprised 
one is essentially based on a notion of distance. St.ll refernng to '''9;^2(^). let us assume a P 
of tv.0 components Al and A2 for consistency wrth the Pr°^°yP« ° ^ W; (P2-A2)^ In this case, 

made for instance using the mere criteria of the Eucl.d.an d'S^n^^^^ AP ^e^AP - PI A ) ( )^ ^^^^^^ 
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learning, the value of radius r is generally given by a default value rO referred to as the maximum influence field MaxlF 
as illustrated in Fig. 2(A), which is generally defined arbitrarily and empirically. However, other distance calculation meth- 
ods are available, for instance the so-called Manhattan or city block distance (L1 norm) and the square distance (Lsup 
norm), which would produce zones of different shapes. The L1 norm would result in a lozenge while the Lsup norm 

5 would result in a square in the two dimension feature space of Fig. 2(A) instead of a circle. 

If we still consider prototype vector P illustrated in Fig. 2(A), let us assume that an input vector A still representing 
the same capital letter "E" but printed with a different type font is now presented to the system user for recognition. If 
input vector A falls within circle Z. it will be thus considered as "similar" to prototype vector P, and in turn, will be labelled 
with the same category C. It is important to notice that the assignment of the category to the input vector by the system 

10 is performed during the recognition phase. If now. input vector A falls outside the circle Z, it will be considered as " not 
similar" with regard to the prototype vector R In this case, the category C cannot be assigned (or associated) to it by 
the system. It is then recommended to have input vector A stored by the user as a new prototype vector with the same 
category C. To that end, in a subsequent learning phase of the system, the user presents again the input vector A with 
its category C to the system. The system stores input vector A as a new prototype vector P' with the same category C 

15 assigned thereto, to provide the extended zone illustrated in Fig. 2(B), comprised of circles Z and Z', which now delineates 
the category C. 

Let us assume now that an input vector A representing capital letter "F" is presented to the system and falls within 
circle Z of prototype vector P. Because, letter "F" obviously belongs to another category according to the definition thereof 
given above, the category C of prototype vector P cannot be assigned to it by the system. As a consequence, circle Z 

20 as originally drawn has now to be retracted not to cover this point A. In others words, the radius rO of the circle encom- 
passing prototype vector P needs to be reduced if the user decides to store this input vector A as a new prototype vector 
P". This step of retraction is part of the so-called "reduction process" which is an essential aspect of such a conventional 
character recognition system. Once input vector A has been stored as prototype vector P", the new radius of circle Z is 
obtained by reducing the initial radius rO = MaxlF to a valuer inferior or equal to distance PP". This valuer also constitutes 

25 the radius of circle Z" ( r" = r ). This new or current radius value r of prototype vector P is usually referred to as the actual 
influencef ield AIF. The two prototype vectors P and P" with their respective associated categories C and C" and influence 
fields r and r" are illustrated in Fig. 2(C). It is to be noted that a minimal value of the influence field, referred to as the 
minimum influence field MinlF, is also defined. Under no circumstances, the actual influence field (AIF) of a prototype 
vector may have a value lower than MinlF. 

30 Fig. 2(D) shows a two dimension feature space comprising three prototype vector P, P' and P" with their respective 
influence fields r, r/ and r" and associated categories C. C, and C". Let us consider now an input vector A is presented 
to the system for recognition, the system is able to calculate the distances AP = D , AP' = D' and AP" = D" and then 
to determine the minimum distance Dmin among them. If input vector A falls within one circle, e.g. circle Z (Dmin < r), 
it will be recognized by the system and the category C will be associated to it. Assuming now input vector A falls in none 

35 Of the circles Z, Z' and Z". as illustrated in Fig. 2(D), the input vector is not recognized and the category is not assigned. 
If the user decides to store input vector A as a new prototype vector, he will present it again to the system with the 
adequate category during a subsequent learning phase. As a matter of fact, the user decides which category will be 
assigned to the new prototype vector, for instance any of category C. C or C", or a new one. If the user decides to assign 
the category of the closest prototype vector (based on the calculation of the minimum distance Dmin), the influence field 

40 of the new stored prototype vector will be a value inferior or equal to Dmin if Dmin < MaxlF and MaxlF if Dmin >= MaxlF. 
In the case shown in Fig. 2(D), this minimum distance Dmin corresponds to distance D = AP . Finally, if input vector A 
falls within an overlapping zone, i.e. a common zone between two circles (not shown), the user will not only determine 
the right category to be assigned to the new prototype vector, but may have to perform the adequate reduction of their 
respective influence fields, so that one prototype vector (or the two prototype vectors P' and P") is excluded from the 

45 recognition. 

Note that, although Figs. 2(A) to 2(D) show an input vector A comprised of two components A1 and A2, it must be 
understood that for generalization, an input (or prototype) vector will be considered as being comprised of n components. 

As a matter of fact, components A1 , A2 An represent the most general definition of the input vector A. In this 

assumption, the circle Z illustrated in Fig. 2(A) may be then represented by an hypersphere in a n-dimension feature 
, 50 space. The computed distance will be the distance separating the center of the hypersphere representing the stored 
prototype vector and the point representing the input vector. The MaxlF value will correspond to the largest radius of 
the hypersphere at initialization. Similarly, the MinlF value will correspond to the smallest radius of the hypersphere that 
is allowed in the course of the reduction process. For distance calculation methods that are different of the Euclidian 
method mentioned above, the equidistant points are no longer placed on an hypershere but on a polyhedral volume. 
55 However, said polyhedral volume will be referred to as an hypersphere for sake of simplicity. 

On the other hand, any component of the input vector, which represents a certain analog value, is coded on m bits, 
i.e. it may be depicted by a binary word of the type aO ... am-1 . For instance, referring again to the two component input 
vector A representing capital letter "E" of Fig. 1 , wherein the first component A1 is equal to 13 and assuming m = 4, A1 
would be then represented by the binary word consisting of aO = 1 , a1 = 1 , a2 = 0, and a3 = 1 . i.e. A1 =1101. 
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AS a final result, one can imagine a computer-based chara^^^^^ 

matically proceed to that comparison of .fP^^ ^^^^f,^;^^^^^^^ 
determine theappropriatecategory or class(.f any thatmustbeass^^^^^^ 

"Lmputersuseneuralnetwor^constructedbyasse^^^^^^^^^^ 

mimic the nervous systems o1 living bodies. A ^"^a' "^^^^^^^ „ l^^,,^, „eural net- 

thereof and it is constructed by interconnectmg an ejrtremely •^'^'^ "^'j'^/^^J^^ ti^e is very short. Further- 

works, the synapses execute calculations .n a parallel ^^J'J^^^^^ Ivior of synapses and connection 

more, the functions of biological neural networks are realized f^^"^"^ the be^ P 
states therebetween during the learning, ^e;'^-' ^7^;^^;=^^^^^^ processus is con- 

voice recognition, process optimization robot control a^ fi^'are pT^Sm th^^^^^^^^ time is large. Neural com- 
ducted sequentially in accordance w-th '"struct ons of a «)^^^^^^ 9 ^^^^ processing 

puters have the special characteristic of being able to reahze [""^^Z ;",^^^^^^ derated while conducting 

procedures which are difficult to place in formal rules When "^"T' 
normal learning, even if the functions which are desirable change o^^^^^^^ 

in addition, neural networks that are implemented ^^^^'l^^"^^^^^^^^^^ by simply replacing the 

base neuron circuits, should a failure occur m one of ^^-^^ f^^^ ° Dol^ble to realize neural networks offering a 
failed neuron circuit with another normally functioning^ in cases whS v^^^ sei^conductor chips are used, 
high resistance to flaws, an aspect of signrficant ^'^^''^''^^'^T^Z^^^ Basic Function (RBF) 

Different neural network architectures may be ^"^''^'""^^ basis functions" by M 

technique, such as described in the article "A h-Qh Performa^^^^^^ S^Cas ul^^^^^^^^ neural network 

Holler et al, Microcircuit Applications Conference Nov^9-12 f 2, Us Vegas^^^^^^ ^^^^ .^p^^ 

of the RBF type consists of a three layer structure. The '^^^ir'^^"^^^^^^ The third layer or 

layer or input neuron layer. The second or hidden layer is form^ f yJ'!j:';Xe^^^^^^ Each neuron circuit has 
neuron output layer utilizes the outputs of the neuron circuits of '^^f^^f '"^"^^^^^^ to the components 

weight coeLents (currently designated in the f,:,^^^^^^^^^^^^ in parallel 

of the stored prototype vector. The input signals available °" ^l^^-f ^^^-^'^^Jj ^"^^^^ i„ ,4 comparison mentioned 
to all the neuron circuits of the second layer for P^-f^^'^a^/]^^^^^^^^^ of the neural network and to 

farertrnro?^^^^^^^^ ^^^^^ — ^ ^ '^^ 

'ns?srrrsu°r=^^^^^ 

t6 Mo'of \h RBF type. fjst laye^ts^ ^ ^ 

comprised of two components Al and A2. This first layer ^ojaiiy ir^ie ^^.^^ 

second layer. Each neuron circuit of the second '^.Vf J^^^^^^^^^^^^^^ ° store the prototype vectors (one 

layer. During the learning phase, the neuron c|rcuits of the second aye^^^re "sed to st p ^^^^^^ 

prototype vector stored per neuron circuit) in a WW "^"^^^^^^^ '° random The neuron circuits are said to 

network 2 is used for the first time, all the weight memories have a contente at random^ ^ 
be "free". As soon as a prototype vector is stored in a "^^^^^^^ I ^TteS^ S egS has been assigned 
neuron circuit of the second layer and an output neuron of t^i.rd laye J^^^^^ free Lnd ifsS "engaged". For instance, 
to it, this neuron Circuit Which has thus "learner' IS ^^^^^^^ 

neuron circuits N2, N5 and N8 (which are associated to the same category o^in o g y 
engaged. Similar reasoning applies to other neuron c.^^^^^^ 

has been mentioned above that the value o the drrmsfances it is allowed to reach a 

course of the reduction process dunng the learning ^^1''°'^^^';^^;^^^^^^^ process, this neuron circuit 

value equal to or less than the MinlF value, and ^.s appens in thecou se ^^^^^J^P^^ .^^^gory has 

is said to be "degenerated". Neuron circurt N10 is stiH ^^^^^f^^^f^^^^^^^ 

been associated to it. the neural network 2 of Fig. 

The feature space depicted in Fig. 3(B) nas oeen aesia / examole of a two-dimension 

3(A) for illustration purposes (only neuron circuit N10 is not ' f J.^^f ^J^^^^^ sS in neuron circuits N1 
feature space illustrating the influence "'^^ P^^^^^^^ C1 . C2 and 

toN9.Theyareorganizedinthregroupso^2^ 

Seror;r.e^^=A^ 

rg?(B%ring^r^^^^^^^^ 
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During the recognition phase where an input vector is presented to the neural network 2 for comparison with all the 
prototype vectors of the feature space, each neuron circuit calculates the distance between the said input vector and its 
stored prototype vector, as discussed by reference to Fig. 2(D). tf the input vector falls within the influence field of a 
determined prototype vector, the category attached to this determined prototype vector is assigned to the input vector. 

5 The same reasoning applies if the input vector falls within the influence fields of several prototype vectors pertaining to 
the same category. In both cases, the input vector which has been recognized by the neural network with a single 
category is said "identified". If the input vector falls within the influence fields of at least two prototype vectors belonging 
to different categories which overlap, the input vector is at least twice recognized but it is not identified and the input 
vector is said "undefined" or "uncertain", because the category to be assigned (or associated) thereto cannot be deter- 

10 mined. In all the above cases, the corresponding neuron circuits which have recognized the input vector are said to have 
"fired" or "triggered", and a fire signal F is set active (F = 1). If the input vector falls within the influence field of none 
prototype vector, none neuron circuit of the neural network fires (each fire signal F is inactive (F = 0). It is worthwhile to 
distinguish between data that are generated at the neuron circuit level labelled "local" or "neuron" data and data gener- 
ated at the neural network level that are labelled "global" or "neural" data. Local data first include result data (e.g. a 

15 neuron circuit which fires generates a local result fire signal) and status data (e.g. a degenerated neuron circuit holds a 
degenerate status) referred to hereinbelow as result data. Local data further include output data (e.g. distance or category 
data). Likewise, global data include global result data (e.g. a neural network which identifies an input vector generates 
a global result identified signal) and global output data (e.g. the minimum distance Dmin). Local data which are repre- 
sentative of the local reaction of an individual neuron circuit to the presentation of the input vector are therefore "con- 

20 solidated" at the output neuron level to produce said global data. 

Neural networks of the type illustrated in Fig. 3(A) have been extensively used in the industry so far. However, none 
of the neuron circuit architectures known to date appears to be really satisfactory and generally exhibit a lot of limitations - 
and/or inconveniences. '"^^ 
First of all, conventional neural networks have none or limited cascadability. By cascadability, there is meant the 

25 capability to connect an unlimited number of neuron circuits to form a complex neural network incorporating as many 
neuron circuits as required by the application. Because neural networks are usually implemented in VLSI semiconductor 
chips, consequently it would be therefore highly desirable to cascade as much neural chips as desired with the minimal 
additional circuitry, to finally produce such a complex neural network. Unfortunately, this flexibility has not been made 
possible so far. As a result, the cascadability of standard neural chips/modules is very limited, because assembling large 

30 quantities of the same is not an easy task and generally requires a significant number of external components. Moreover, 
should such external components be required, the number of neuron circuits in a complex neural network would not be 
transparent for the user any longer. This transparency is a factor of paramount importance, in particular in the learning 
phase. 

Next, conventional neural networks that have been designed to date are not autonomous, i.e. the obtainment of^any 

35 global result at the neural network level, mandatorily requires the supervision of a digital computer, typically a micro- 
controller or a dedicated micro-processor for overall data management. For instance, one may refer to US- A-51 65010 
(Ref. D2) and more particularly to Fig. 23, which illustrates a micro-controller providing the supervision to a plurality of 
neuron circuits which all together form the neural network. The neural system described therein is organized with the 
same parallel architecture as in conventional micro-controller. Between the neuron circuits and the micro-controller, the 

40 data are exchanged via the data bus while the addresses are circulating on the address bus as standard. In conventional 
neural networks, the neuron circuits are thus totally passive and communicate only with the micro-controller. In other 
words, there is no data communication or exchange between the different neuron circuits which form the neural network. 
In addition, because said computers are software operated, either the recognition or the learning phase results in a 
lengthy and complex operation. Still another consequence is that with conventional neural chips, the number of input/out- 

45 put pads is dependent of the number of neuron circuits integrated therein because of addressing requirements. Same 
reasoning applies to the input/output pins of the electronic neural modules incorporating such neural chips. 

On the other hand, the number of categories that are available in such conventional neural networks is limited. For 
instance if we now refer to the neural network described in US-A-4326259 (Ref. D3), the neuron circuits are arranged 
in a column to feed the horizontal lines of a PLA, whose vertical lines supply the categories. As apparent from Fig. 8 of 

50 that reference, it is clear that for many reasons, the number of categories cannot be extensively increased, in particular 
because the result must be interpreted by the user. The global information relating to the formal identification of the input 
vector by the neural network is not directly generated. If several neuron circuits fire, the user has to interpret the results. 

Likewise, if we consider a category as a label attached to each output neuron of the neural network 2 of Fig. 3(A), 
it is to be noted that a similar label does not exist at the neuron circuit level. This is another limitation of conventional 

55 neuron circuit architectures, should certain neuron circuits need to be prevented to participate to the recognition phase 
of a determined family of input vectors. This approach would also permit to organize the neural network either as a single 
network or as subsets thereof that would be programmable by the user. 

Finally, the learning is not self-made, but generally the weights are determined separately by the micro-controller 
and subsequently loaded in the neuron circuits when the micro-controller estimates the learning phase completed. As 
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a consequence, the recognition and the -earning phases are c^a.y sTiilltirJir^^^^^^^^^^^ 

training is accomplished by adjusting the weights which "^^^''^ then a rriathematical 

through the neural network and the output and the output signal 
computation will determine how the weights should be ^diust^^ . JS^^s such as taught in US-A-52221 93 (Ret. 

an example related to ^^^.^^^^^^T^;^^^^^ the first neuron circuit to get the 

vectors stored in neuron circuits N1 to N9 Typ caiiy,tne micro comparison between the two 

distance it has computed, then it interroga es '^^^^"^ ^^u^'^n s^^^^ J successive iterations until the last 
distances to hold the lowest Tm ^^^^^^ input vector and all the prototype vectors 

neuron circuit has been interrogated. The '"'"''^"'^^^'^^^/^.^^l^^g^^^^^^^ the reduction process mentioned above, 
is thus only determined at the end of the P^^^^^^^'^nrf^c^^^ the neuron 

A similar technique is applied during the reduction P^^^f/^ f "^^^^^ ^^^J^^-^^ the adequate category In all cases, 
circuits which have wrongly fired until ,t remains only "^^^ "^^^^^^^ and thus mcluding a number of lines of instruc- 
a dedicated software program based upon a complex ^'9°^^^^^^^ such, the sort process is really 

tions. is required to perform the total sequence of '"^^^^^^a^^^^^^ of the neural network 2 of Fig. 

time consuming. Because of the lack ^f.^'^f '^^^^^^ the neuron circuit level and the global 

3(A), the potential correlations which exist between all ^ consequence, the conventional neural 

result signals at the neural "-^^-V'^.t inn to h^usTalto t^e g^bal^^^^^^^^^ addition, the number of cate- 
network of Fig. 3(A) only provides limited '"^'^f ""^^^i^ulvrmifX^^^ neuron hardware construction. Most, 

gories that are available at the output neuron level IS obv^^^^^^^ 

^a^^Sr^aTdt^^^ 

.91 IMMARY O P THF INVENTION 

,n a neural network comprised of a plurality of N 
distance D coded on p bits between an ^^^J^^^^^^^^^^^^ 

according to the present invention there is «^'.=^'°f ^^^^^^^^^^^^ distance among said calculated 

placed in a neuron circuit. This aggregate circuit 'f^^f ^^^^^^^^^^^ to a first variant, any search/sort 

distances and then to sort the remaining ^" '"^^^^^^^^^^ a column direction. Each base unit 

circuit is comprised of p elementary base units '^^""/^^^f .^^^^^^^^ and in turn, generates 

receives three input bit signals: ^ distance bit signal an e^^^^^^ The distance bit signals of the 

two output bit signals, a local (or neuron) °"*P"V.='9"^' 
same bit rankare applied to said baseunitsaccordingalinediredion^Asac^^^^ 

circuit are organized in a matrix. The exclude °" J" distance bit signal If the 

next base unit in the column. The local output s^nal is ."^^^ to 1) if the feedback signal is 
exclude input signal is inactive (equal to 0). The ^'^^^l" °Zd.^^^^^^^^^ 

complementary to the local output signal signal or if '"P^j^J^^^^^^^^^^^^ a!, the local output signals generated 

to the signal obtained by performing an OR function, ^" zeroes in the distance bit signals, 

by the base units of a determined line. The search process is on he sea ch o ^.^^ 

from the MSB's to the LSB's. If a zero is found in a Jf^^J^^^^^^^^ i.e. the searched min- 

excluded from the following search. The process 's continued ;™ ^ J,^ ,,cuit is only adapted 

imum distance, that is finally available at the °"*P"1°* ^^f °^ s2rch/sort circuit can be signif i- 

(and last) distance is available. The same reasoning applies to the category type ot oa 
nRIECTSOFTMF INVENTION 
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It is another object of the present invention to provide a search/sort circuit placed in each neuron circuit of a neural 
network that is adapted to cooperate with corresponding search/sort circuits of the other neuron circuits for a fast deter- 
mination of the minimum distance Dmin among all the distances calculated by the neuron circuits. 

It is another object of the present invention to provide a search/sort circuit placed in each neuron circuit of a neural 
5 network that is adapted to cooperate with corresponding search/sort circuits of the other neuron circuits for a fast deter- 
mination of the minimum category Cmin among all the categories calculated by the neuron circuits. 

It is still another object of the present invention to provide a search/sort circuit placed in each neuron circuit that is 
designed to cooperate with similar search/sort circuits of other neuron circuits to sort the remaining distances in an 
increasing order after the minimum distance Dmin has been determined. 
10 It is still another object of the present invention to provide a search/sort circuit placed in each neuron circuit that is 
designed to cooperate with similar search/sort circuits of other neuron circuits to sort the remaining categories in an 
increasing order after the minimum category Cmin has been determined. 

It is still another object of the present invention to provide a search/sort circuit placed in each neuron circuit that has 
a slice structure adapted to be aggregated with sorter circuits of other neuron circuits for full parallel processing of the 
75 distances and, at least theoretically, without a limit on the number thereof. 

It is yet another object of the present invention to provide a search/sort circuit placed in each neuron circuit of a 
neural network that only requires a minimum number of devices for maximum circuit integration. 

The novel features believed to be characteristic of this invention are set forth in the appended claims. The invention 
itself, however, as well as other objects and advantages thereof, may be best understood by reference to the following 
20 detailed description of an illustrated preferred embodiment to be read in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS ^.t^ 

Fig. 1 shows the capital letter "E" embedded in a 9x8 matrix of pixels. 

25 Fig. 2 which is comprised of Fig. 2(A) to Fig. 2(D) aims to show different examples of the implementation of a 
prototype vector in a two dimension feature space. Fig. 2(A) shows a single prototype vector and its influence field 
illustrated by a circle; Fig. 2(B) shows two prototype vectors having the same category C; Fig. 2(C) shows two prototype 
vectors pertaining to different categories to illustrate the principles at the base of the reduction technique; Fig. 2(D) 
shows three different prototype vectors pertaining to different categories to illustrate the notion of distance. 

30 Fig. 3 is comprised of Figs. 3(A) and 3(B). Fig. 3(A) shows a conventional RBFthree layer neural network architecture 
comprising ten neuron circuits adapted to process two-component input vectors for their classification into three cate- 
gories. Fig. 3(B) shows an example of what could be the feature space corresponding to the neural network of Fig. 3(A). 

Fig. 4(A) schematically shows the block diagram of a first architecture variant of the neural semiconductor chip when 
the latter is designed only for stand-alone usage and the neural network that is incorporated therein according to the 

35 present invention. 

Fig. 4(B) schematically shows the block diagram of the preferred architecture of the base neural semiconductor chip 
when the latter is designed for operating either for stand-alone usage or with other similar neural chips in a multi-chip 
environment and the base neural network incorporated therein according to the preferred embodiment of the present 
invention. 

40 Fig, 5 shows the schematic block diagram of the preferred architecture of the innovative neuron circuit of Figs. 4(A) 
and 4(B) still according to the present invention to illustrate its essential constituting functional blocks and main inter- 
• connections therebetween. 

Fig. 6 shows the schematic block diagram of the status and control logic circuit 18 of Figs. 4(A) and 4(B) which 
generates the internal control, selection and address signals that are required for the operation of all the blocks of Fig. 5. 
45 Fig. 7 shows the schematic block diagram of the IF circuit 350 of Fig. 5 which is comprised of a register and a 
multiplexer. 

Fig. 8 shows the schematic block diagram of the Dmin determination circuit 500 of Fig. 5 which is comprised of a 
multiplexer, a logic circuit and an innovative search/sort circuit. 

Figs. 9 and 10 show two architecture variants of the base neural chip of Fig. 4(B). 
50 Fig. 1 1 shows the elementary module formed by assembling a plurality of the Fig. 4(B) base neural semiconductor 
chips connected in series to illustrate the full cascadability capability of said neural chips of the present invention within 
the limit number permitted by the semiconductor technology. 

Fig. 12 shows the complex module formed by assembling two elementary modules of Fig. 1 1 at the cost of only a 
very small additional circuitry. 
55 Fig. 1 3 shows the flow-chart of the essential steps of the initialization phase of the base neural network. 

Fig. 1 4 that is comprised of Figs. 14(A) and (B) shows the flow-chart of the essential steps of the recognition phase 
of the base neural network. 

Fig. 1 5 shows the flow-chart of the essential steps of the learning phase of the base neural network which comprises 
the reduction process and the engagement process , 
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Fin 1 6 Shows the schematic circuit construction of the matching circuit 1 50 of Fig. 5. , 

of Fig. 4(B) ^jg^ R/VV memory circuit 250 of Fig^5 

Fig. 18 IS comprised of Figs. 18 (A) (B and (Or Ma ) ^^id circuit 250 during the 

which is basically organized around a RAM '^.^'^^^y/.^^^.^iJ^^^^^^^^ during the recognition phase for the 

"""fS aJ'SL the d«a»ed cons»u«lon o. tira part 0. »e adder ol Bg. 20 which generates intermediate M 
signals to be used in the se»™l P«rt«^weol. ^^^.^ ^„g^a|„„ ^ , xOR circuit «ith 

diJ^.^ro3n^^"Fir-,SSSi52-t'..Uci,^^^ 

c^SiJ^n ,n F^^ ,8 are extensively '^!\'^<^^;^'^TS^>'^''^^<^ sun,h«»gnals. 

'"^T.'fs;?n:ti::irsssr~^«^^ 

result and status signals at «>e neuron ""'I ^„ determination circuit 500 ol Fig. 8. 

(assuming the latter is only comprised of Fig. 29 further including storage 

Fig 30 schematically shows another variant of the fictitious aggregate circuit or ng 
elements to allow the remaining distances to be sorted in the '^creasing order^ 

^^jijr:ri:^r.~r«r^^^^^^^^^ 

™'"^g^rSrows^estru«ureo,*eln,erneu,oncommun,ca«c.s^^^^^ 

"TrS^hows^he detailed implementation of the driver circuits for the first bit of the global output signals in case 
Fin -ict illustrates the aDplicatlon of the inter-neuron communication system approacn ot tne pres.e 



55 

unit 



Fig. 41 Shows the implementation of a dummy search/sort circuit in each base neural chip of Fig. 4(B) to generate 
the MaxlF signal as a local output signal. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 
BRIEF DESCRIPTION OF THE ZISC CHIP ARCHITECTURES 

5 MNTRODUCTION * 

According to the present invention, the prototype of a neural chip has been manufactured in a conventional 1 pim 
CMOS technology. The number n of vector components ranges from 1 (the minimum number) to 64 (the maximum 
number). The number m of bits for coding each component is 8. The input data interface is performed through a 16-bit 

10 bidirectional bus. The L1 and Lsup norms mentioned above are available for calculating the distance with an accuracy 
of 14 bits. The maximum number of contexts and of categories is 127 and 16384 since they are respectively coded on 
7 and 14 bits. MaxlF and MinlF which correspond to the radius values of the largest and smallest influence fields are 
also coded on 14 bits. At the initialization of the ZISC chip 10, these values are respectively equal to 010 ... 00 (I.e. 2**13 
= 8192) and 00 ... 10 (i.e. 2**1 = 2). They are default values determined empirically and arbitrarily. Thirty six neuron 

15 circuits have been integrated in the silicon substrate for a total of about 400000 transistors in the chip. These neuron 
circuits that have an innovative architecture generates result and output signals. To emphasize the fully hardwired and 
autonomous aspects of this neural semiconductor chip, it will be referred to below as the ZISC chip. ZISC (a registered 
trade mark of IBM Corp.) is an acronym for Zero Instruction Set Computer (or Classifier). As a consequence, the ZISC 
chip fundamentally distinguishes from the standard CISC or RISC computer chips because it does not require any set 

20 of instructions for its operation. This ZISC denomination is also applicable to the said innovative neuron circuits, to the 
modules built from assembling said neural chips and finally to the neural networks incorporated in said neural chips or 
modules. 4.5 

* THE BASE ZISC CHIP ARCHITECTURE * 

25 

Now turning to Fig. 4(A), there is shown the schematic block diagram of the architecture of a first variant of the base 
ZISC chip of the present invention referenced 10. The chip architecture includes a plurality of innovative neuron circuits 
of the type generating local (or neuron) result and local (or neuron) output data and incorporates a neural network still 
according to the present invention. The single chip architecture illustrated in Fig. 4(A) is tailored for stand-alone appli- 

30 cations. The innovative neuron circuits are included in blocks referenced 1 1 -1 to 1 1 -N which all together form neural unit 
1 1 (#). As apparent from Fig. 4(A), all the neuron circuits 1 1 -1 to 1 1 -N have a serial connection so that they are structured 
as a chain. Assuming in this preferred embodiment that all these neuron circuits are identical, an illustrative neuron 
circuit will be referred to by numeral 11 (or 1 1-1 as the case may be). The preferred architecture of neuron circuit 1 1 will 
be described hereinbelow in more details by reference to Fig. 5. In the following description, numeral 1 1 designates the 

35 architecture of the neuon circuit as well as the neuron circuit itself. Similar notation applies to the architecture of the 
ZISC chip 10 and to the ZISC chip 10 itself. In Fig. 4(A), the neural network is referenced 1 1(A). It basically consists of 
neural unit 1 1(#) and inter-neuron communication system formed in the single ZISC chip 10. According to the present 
implementation of the variant illustrated in Fig. 4(A), the inter-neuron communication system is comprised of block 12 
and an on-chip common communication bus labelled COM*-BUS which will be described in more details hereafter. All 

40 the blocks of Fig. 4(A) are comprised of circuits that can be easily integrated in a silicon substrate to form a VLSI IC 
chip. The ZISC chip 10 does not specifically require a micro-controller to perform the supervision of the neural network 
1 1 (A) operation as it is currently practiced in standard neural chips. As a consequence, none instruction set is required. 
A micro-controller (or a micro-processor) could be used in conjunction with the ZISC chip 10, but its role would be limited 
to present the input vectors (and categories) to the neural network 11(A) and get the global response therefrom. For 

45 convenience, these operations will be referred to as the WRITE and READ operations hereafter. However, it must be 
understood that because of the improved architecture of the ZISC chip 10 of Fig. 4(A), these operations could be per- 
formed by the user as well, through manually operated switches. The connections between certain blocks of ZISC chip 
10 and the micro-controller or the user (not shown) are illustrated by the 16-bit bidirectional input data bus labelled 
INDATA-BUS (connected to the I/O bus of the micro-controller if any) and by a SEL7CTL-BUS which transports selection 

50 and control signals. The conventional input 16-bit register (DATA register) which latches the data appearing on the 
INDATA-BUS at every cycle has not been shown. The receiver, driver and receiver/driver circuits which interface the 
ZISC chip 10 with the external world have been schematically represented by black boxes respectively labelled RR, DR, 
or RR/DR as the case may be. They all are conventional circuits for the circuit designer. 

Block 13 is a 8-bft register which is loaded from the DATA-BUS with data representing the components of the input 

55 vector. This register is optional in asynchronous mode and is only required in the synchronous (BURST mode) of the 
ZISC chip 10 operation. 

In addition to the input vector data, different set-up parameters (such as MaxlF and MinlF values) and input data 
(such as the input category, input context and norm) are also latched in dedicated registers of the ZISC chip 10. 
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„om> and context data. The bus that Is outputtad Iron, 'r^^''^^'^,^^^^,^^^^ coniunction ««h Fig, 5. 

purposes. Most ot these signals are generated n a SP«'* 2^S^™lc Circuits that are rsfluired to a proper 

SSi^"r^S;Tarm"cC:r2 S - he desonhed ,n n^e de«iis 

!zrj:=T^;:t;r,iXie'^n^sroHh'ez,scch,tode.^^^^ 

limitatively interpreted. .^♦„t«.= the Wnrk rliaaram architecture of an individual neuron 

Let us now consider Fig. 5 which =<=herr.at.cally Hlustrates the t^^^^^^^ the NO/CXT-BUS is simultaneously 
circuit referenced 11 according to the present ''^fZ^taZ l^lo^^^^^^ 

applied to a block 100 referred to as the local JJ'^ ^^^^^^^^^^^ The NO/CXT-BUS transports a 

a conventional matching or compare circurt made °* ^0*^^^"^ °n,or^a,Sco^^^^^^^^^ 
l-bitinputnorm signal anda7-bit input contextdata^Oncet^^^^^ 

referred to hereinbelow as the multi-norm <^'!*«"=«^^^^^^^ or the user has decided that an input vector 

According to the present invention, as soon °^ ^1 „„ron circuit 11 or to proceed to the 

should be learned by the neural network 11(A), t^«.d«'=;='°" °! ^^^^^^^ neural network 11(A) 

reduction of the AIF value of the engaged neuron circuits J^^^^^^^J^^^-,^ i3 automatically loaded from 
itself. At the end of the engagement P^^f iCal^eS^^^ the main NO/CXT register 

the main NO/CXT register 17 of Fig. 4(A) ^'^f .f„^0/CXT BU^^ 

t;rcaro:m";tr^^^^^^^ 

and context (CXT) data that are held in main NO/CXT '^'^^^'JJ..^^^..^ ^.^^ to differentiate different types of 

. i.uf:e^r.^=h=dr-=^^ 

the capital characters would be loaded with a local context ^^J^'^^J^ 1^^^% ^^,^3^ data equal to a second 

circuits having in charge to recognize lower case Characters w^^^^^ 

,5 binary value. As a consequence, the local/inpu context the context value stored 

with a determined input context and to inhibit all °thers At each recogn^^^^^ ^.^ ^^^^^^ 

in the local norm/context register 100 of '^^^^"i^,^^^ This comparison is per- 

register 1 7. If found identical, the neuron circuit will be se'ect«^- J^hj^'^^ '^^^ ^^^^^^ Selection) to select 

formed in matching circuit 150 whose role is tS ma tc^^^^^^^ receives the CXT signal 

or not a ^-^i"" by norm/context register 100 via 

via the context port on (CXT BU S) of the^^^^ ^^^^^^ ^^.^ ^,^^^,3 signal is set 

the context portion (Cxt-BUS) of the No/Cxt ^'iJ^/^";" ;/';"' ^ its of z,sc chip 10 as it will be explained 

active to enable said neuron circuit to ^^"'^f J^*!*^^^^^^^^^ for enabling or disabling them 

later on. Signal NS will be applied to certain blocks of neuron c-rcurt a chiterture ^ ng a previously 

as the case may be. During the recognition phase, a approach of the 

engaged) and which is selected by ^^^^^^S f «n^^^^^^^^^ algle neTal network or as an arrangement of 
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On the other hand, the local norm signal No determines how to calculate the distance between the input vector A 
and the prototype vector B stored in the weight memory of the neuron circuit 11 as it will now be described. Still according 
to the present invention, the local norm signal No selects the desired calculation algorithm through adequate circuitry 
implementation inside evaluator circuit 200 (the ZISC chip 10 is based on a totally hardware solution). In the present 

5 instance, each neuron circuit 11 of the ZISC chip 10 can calculate either the MANHATTAN distance (LI norm) i.e. 
D = sum (abs (Ak-Bk)) orthe SQUARE distance (Lsup norm), i.e. D = max (abs (Ak-Bk)) wherein variable k takes every 
value from 1 to n. Note that "abs" is an usual abbreviation for "absolute value". According to the disclosed ZISC chip 10 
implementation where the local norm and context data are respectively coded on 1 and 7 bits. For instance, the most 
significant bit (MSB) of the No/Cxt-BUS will represent the local norm and the remaining bits will represent the local 

w context. At initialization, this local norm bit No is equal to zero in order to compute the MANHATTAN distance by default. 
On the other hand, although 7 bits are used to hold the local context data, only up to 127 different context values are 
permitted, because the zero value is reserved. In the latter case, all the neuron circuits of the ZISC chip 10 are uncon- 
ditionally selected. In other words, the above mentioned comparison between the local and input context data is not 
performed. 

15 The DATA-BUS is applied to one input of the multi-norm distance evaluator circuit 200 and to a block 250, typically 
a Read/Write memory circuit. According to the present implementation of the ZISC chip 10 of Fig. 4(A), the Read/write 
memory 250 essentially consists of a 64 words x 8 bits RAM macro usually referred to as the weight memory. To have 
a R/W memory circuit adapted to store the components of the prototype vector associated thereto in each neuron circuit 
1 1 instead of a single R/W memory circuit common to all the neuron circuits of the neural network, is a significant feature 

20 of the present invention. The bus which connects the R/W memory circuit 250 to the other input of evaluator circuit 200 
is labelled RAM-BUS. During the recognition phase, the R/W memory circuits 250 of all the neuron circuits 1 1 -1 to 1 1- 
N are addressed in parallel at the same time and with the same component address by block 1 8 via the I ADD/CTUBUS. 
A control signal labelled RS is applied to R/W memory circuit 250. The RS signal is an enable signal that allows to'store 
the components of the input vector in the R/W memory circuit 250 of only a determined free neuron circuit, for instance, 

25 thef irst one in the chain. This first free neuron circuit will be referred to hereinbelow as the "ready to learn" neuron circuit. 
For the neuron circuits that have been previously engaged i.e. whose the R/W memory circuit 250 already contains the 
components of a prototype vector, the RS signal that is generated by these neuron circuits is such that writing therein 
is no longer permitted. Loading the input vector components in the R/W memory circuit 250 of the ready to learn neuron 
circuit during the recognition phase is another significant feature of the innovative neuron circuit architecture according 

30 to the present invention. For power consumption saving, the input vector components are not loaded in the RAM memory 
circuit of all free neuron circuits. Preferably, only the evaluator circuit 200 of engaged neuron circuits calculate the distance 
between the input vector A presented on the DATA-BUS and the prototype vector B stored in said R/W memory circuit 
250. Depending on the value of the local norm signal No which is applied to evaluator circuit 200, the distance D to be 
calculated will be either the MANHATTAN distance for the LI norm or the SQUARE distance for the Lsup norm as 

35 explained above. However, other calculation methods could be envisioned as well. To increase the processing speed, 
the distance calculation is initiated as soon as the first input vector component is presented to evaluator circuit 200. The 
evaluator circuit 200 generates the final distance D signal on the 1 4-bit DIST-BUS. when all the said components have 
been processed. The last component of an input vector is followed by a control signal LCOMP (Last COMPonent). 
The distance D signal is applied to a first input of block 300 via the DIST-BUS and to the first input of block 350 via 

40 the DIST-BUS- The structure of block 350 is schematically shown in Fig. 7. The signal J which is outputted from block 
350 is applied to the secord input of block 300, 

As apparent from Fig. 7, block 350 basically consists of an IF circuit which is comprised of four-way multiplexer 351 
and a 14-bit AIF register 352. The multiplexer 351 receives the OR-BUS, the DIST-BUS. the MaxIF/MinlF-BUS and its 
own output via the said AIF register 352 to latch the data outputted therefrom. At this stage of the description, it just 

45 suffices to say that the OR-BUS is a 1 4-bit width bus which essentially transports distance (and in particular the minimum 
distance Dmin) or category type of data or signals. During the learning phase. AIF register 352 of neuron circuit 1 1 is 
loaded with a value whose maximum is given by MaxlF via the MaxIF/MinlF-BUS. Since then, its contents is subject to 
changes during the subsequent learning phases whenever required. The AIF value of a committed neuron circuit may 
be equal either to the distance D calculated by the neuron circuit 1 1 . to MinlF (if D < MinlF) or to MaxlF (if D > MaxlF) 

50 at the end of the reduction process. The AIF value which is loaded in the just engaged first free neuron circuit is either 
Dmin (the distance between the input vector and the nearest neighboring neuron circuit). MinlF or MaxlF As a matter 
of fact, AIF register 352 holds the AIF, i.e. a value comprised in the range defined by MaxlF and MinlF (respective upper 
and lower limits). The AIF value cannot be changed during the recognition phase, but only during the learning phase. 
As soon as this lower limit is reached (AIF = MinlF), the corresponding neuron circuit 1 1 is said degenerated. Signal J 

55 is outputted from IF circuit 350 on the 1 4-bit J-BUS. The nature of the J signal is determined by application of adequate 
control signals to multiplexer 351 . 

Now turning again to Fig. 5, block 300 basically consists of an innovative compare circuit which performs the com- 
parison between signals D and J and generates two signals LT (Lower Than) and LTE (Lower Than or Equal) whose 
logic value depends on the result of that comparison. Different cases are to be considered depending on the operating 
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the input vector is not recognized, the neuron circu-t doesn t f -re and ^m^L reduction process, 

nition mode, the LTE signal is not exploited. On the ^^^f^,"?^^"^' I" (j = MinlF ). Hdistance 
comparecircuitaooperforms comparison between thecalcu^^^^^^^ 

D is inferior or equal to the MinlF value , .e. ,f ^ < = .^^Jl:,*^^^^",^^^^^^^ setting a dedicated register 
TeSrSSo^^o^^^ 

a so connected to the OR-BUb, tne omi duo emu ^ r at-rus as the local category data for 

ventional 14-bit register to latch the input category f ^^^P^^.^^f^^^^^^ The bus which is 

subsequent processing. At initialization, the ° '"ff^f °:?^S^;^ local category data or signal 

connected to the output of category register 450 '^.'^'^ ^^jS'lufmlVbe^^^^^^ the lo<5l category data C 

C. AS a result, the input category data CAT appearing on t^e CAT-BUS may bediffe^-J^ o ^ the 

held in category register 450. When, the distincton ""/^^f ^^^^^^^^^^ a determined input 

single term of category data. During the 'earning phase, the category reg^^^^^^^^^ ^^^^ 

category value as soon as the neural networK 11(A) f^/^^^^^f ^1^^°^^ nnovatlve klentHication 
neuron circuit of the neural network has recognized the input ;;^°^,*^3"b^^^^ recognized or not by neuron 

circuit whose a primary function is to make apparent -t^f/^^^^ "^^^"^^^^^^^^^ signals F, DEGOUT 
circuitllduringtherecognitionphase^ldentrf.^^^^^^^^^ 

rthTgrrn™^^^^ 

to the input vector presentation. ^ . ^ recognized the input 

of ident^ication circuit 400 as a result of 

DEGOUT = DEG AND F. The ^^-^-^l^JJ^^^ individual neuron circuit 1 1 to the 

neuron circuit has fired. Signals F and DEGOUT ^[^ '^^^^ ^^^^ P p,aysthe role of a gating signal, as it 
presentation of an input vector to neural network 1 1 (A . In addition ^'f ^' ""^^ J j j^gt have not fired to 
will be explained later on in more details. In particular, it P^«-«"^J*^^°^! ^J^^"*^^^^^^^ 
communicate with the other neuron circuits that have fired via ^^^^"^^^^^^^^^^ Ok signal whose slgnHi- 

bus COM--BUS ment^ned above^FinanyJ^^^^^^^^ 

cation will be discussed later on. The UNC/F'RE^OK signal is^^^^^^^ ^^^^^^ ^.^^^.^ ^ ^ 

^^^rrC "S:::;^^^^^^^^^^^ ^ Hes . .^s.^ design of block 500 which is 

. -ti;;a^e:;rFi^^^^^^^^^^^^^ 

three-way multiplexer 501 . innovative search/sort circu. 502^ and log. circuit 5^3^ T -B^a f^^^'^^^^^^^^^ 
Cat-BUS are applied to multiplexer 501 as '"P"* « J IJf ^^^^^ a local output signal labelled 

search/sort circuit 502 whose other inpu^ '^^''^J^f^ SnifTiuSLS^c^cu^i3 receives three input signals F, RS 
NOUT (NOUT stands for NeuronOUTpu^^^^^^^^ 

50 and CO and generates a selection signal BEL-ECT thans usetuno tne op ^ 

and the role of control f 9-ls RS and CO ^ ^e^'^ ':^^p tr^^h sear^ ciicuit 502 depending on 

local category C signal (via the Cat-BUS) can be directly aPP ^ by-passed or passed-through 

the control signals applied to multiplexer 501 as ^^^^^^/^'/e^^^^^^^^^^ implementation of the 

so that, in particular these signals can be made availaWe Search/sort 

ss ZISC Chip 10, search/sort circuit 502 is '^JP^^^^' ^^^f " circuits of the other 

circuit502of neuron circurtllhasb^en^^^^^^^^ 
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circuits 11-1 to 1 1 -N that are committed and have fired, the minimum distance Dmin among all the distances calculated, 
and then to sort the remaining distances in the increasing order. 

Block 600 is still another further innovative feature of the neuron circuit 1 1 of the present invention. Block 600 essen- 
tially consists of a daisy chain circuit which allows a serial connection with similar daisy chain circuits of the two adjacent 

5 neuron circuits, so that as mentioned above, all the neuron circuits of the ZISC chip 10 of Fig. 4(A) are forming a chain 
like structure. To that end. it is provided with dedicated input and output terminals DCI and DCO. Daisy chain circuit 600 
is organized around a 1-bit register (DAISY REG) which stores a first binary value (e.g. a zero) when the neuron circuit 
1 1 is free and a second binary value (e.g. a one) when the neuron circuit 1 1 is engaged. At initialization, the contents 
of said 1 -bit register is zero. As a result of the particular structure of daisy chain circuit 600, the free neuron circuit that 

10 is ready to learn is easily identified, because it is the first in the chain of neuron circuits 11 -1 to 1 1-N whose DCI and 
DCO signals respectively applied to the DCI and DCO terminals are complementary one to the other (e.g. signal DCI = 

I and signal DCO = 0). In the learning phase of the said ready to learn neuron circuit, a signal labelled ST (ST stands 
for STorage enable) is activated to load a "one" in said 1-bit register of the daisy chain circuit 600 at the end of the 
engagement process. Then, the daisy chain circuit 600 generates a 1-bit signal DCO = 1 at its DCO terminal which in 

15 turn, is applied as the input signal to the input terminal DCI of the daisy chain circuit of the following neuron circuit The 
latter thus becomes the new ready to learn neuron circuit. As soon as signal ST has been activated, the engagement 
processus of neuron circuit 1 1 is initiated. When all the neuron circuits 11 -1 to 1 1 -N of neural network 1 1 (A) are definitely 
engaged, all their respective 1-bit DAISY registers are filled with one's, as a result, DCO-N signal is equal to 1. The 
DCO-N signal may thus be used as a FULL signal. When set active, it means that there is no longer any free neuron 

20 circuit 1 1 in the ZISC chip 10. Finally, daisy chain circuit 600 also generates control signals labelled CO and RS that are 
important to the neuron circuit 1 1 operation. 

As a matter of fact, like most of the other blocks of Fig. 5. identification circuit 400 and Dmin determination .circuit 
500 are controlled by different control signals. Some control signals prevent signal emission on the NR-BUS and/orihe 
NOUT-BUS as the case may be. The aim is to have the NR and NOUT signals outputted only for those of neuron circuits 

25 that are committed and have fired. To that end. a control signal labelled CO is generated from ANDing signals DCO and 
NS. The role of signal CO is thus to specify whether the neuron circuit is committed or not, depending its logic value. 
The CO signal is applied to identification circuit 400 and Dmin determination circuit 500 and is set active (CO = 1) only 
for a committed neuron circuit. Signal CO generated by daisy chain circuit 600 and signal F generated by identification 
circuit 400 are applied as control signals to Dmin determination circuit 500. As a result, thanks to this combination of 

30 the F and CO signals, now only the committed neuron circuits that have fired will be allowed to generate the local 
composite result and output signals NR and NOUT on their respective NR-BUS and NOUT-BUS. Another control signal 
labelled RS (RS stands for Ready to Store) derived from the DCI and DCO signals through an exclusive OR function is 
also constructed in daisy chain circuit 600. As a matter of fact, in neural network 1 1 (A), the first free neuron circuit that 
is ready to learn is the only one which meets the logic relation: signal DCI = DCO. The role of the RS signal is thus to 

35 permit or not the WRITE operation in R/W memory circuit 250. During each recognition phase, the components of the 
input vector are only stored in the R/W memory circuit 250 of the ready to learn neuron circuit. If the neural network 

I I (A) decides to store the input vector as a new prototype vector in the ready to learn neuron circuit, then the RS signal 
of the engaged neuron circuits will be switched to the adequate logic level to prevent any further writing therein. The RS 
signal is also applied to Dmin determination circuit 500 during a particular operating mode (SAVE mode) of the ZISC 

40 chip 1 0 as it will be described later on. 

In summary, as a result of the architecture described in Fig.5, any committed neuron circuit 1 1 that has fired during 
the recognition phase, locally generates a local response to the presentation of an input vector. This local (or neuron) 
response is essentially comprised of two types of data or signals. First, the local result signals F, DEGOUT and 
UNC/FIRE.OK generated by identification circuit 400 that are outputted on the 3-bit NR-BUS and second, the local output 

45 signal (NOUT) generated by the Dmin determination circuit 500 on the 14-bit NOUT-BUS. If the search/sort circuit 502 
of Dmin determination circuit 500 is passed-through, the NOUT signal is either the local distance D calculated by distance 
evaluator circuit 200 or the local category C held in register 450. In the contrary, if the search/sort circuit 502 is not 
passed-through, at the end of the minimum distance determination process, the minimum distance Dmin appears on 
the NOUT-BUS of the neuron circuit which holds the same, while all other neuron circuits are excluded. In other words, 

50 the NOUT signals of these neuron circuits will have neutral values. Because, in the present implementation of the ZISC 
chip 10 of Fig. 4(A). an OR circuit 12 is employed, the neutral value is given by an NOUT signal whose all bits are equal 
to zero. However, it should be noted that several neuron circuits may hold the same minimum distance Dmin value. 

Now turning again to Fig. 4(A). the composite local result signal NR and the local output signal NOUT outputted by 
all neuron circuits 11-1 to 1 1-N of neural network 11(A) are applied to OR circuit 12 via their respective NR-1-BUS to 

55 NR-N-BUS and NOUT-1 -BUS to NOUT-N-BUS. In fact, only the local result and output signals of those neuron circuits 
that are committed and have fired are of significance for subsequent processing in OR circuit 12 (other are neutral as 
explained just above). 

OR circuit 12 basically consists of four dedicated OR sub-circuits in the disclosed implementation of the ZISC chip 
10. First of all, three dedicated OR sub-circuits perform an OR function between each of the neuron result signals, to 
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generate three correspor^ding global result signals i.e. F^ DEGOUr and UNC/FIRE.OK* on^^^^^^^^^ 

ruS. Naming Fi the local result .ire" signal of neuron circurt 1 1 -i (i = 1 to N) of -ra.neW.ork 1 AM e ^^^^^^^^^ 

UNanRE o"kN) S?gnll DEGOUr is thus set aciive (DEGOUT* = 1) when at least one degenerated co--*^ "^^^^^^^^^^ 
are general b, neuron ciraits ll-l » UM ^r "Ch M , (, - 1 » P - "« Ji^^"' ™ ^'^^.^ „„, ^, 

si;!Strarr^;^"c%?.^rL^^^^^^^ 

b COM* BUS The OUT* signal is directly applied as a feedback signal v.a a feedback bus f ^^elled OR-BUS to ^ 
DiT k^vjivi DUO. Mic^w y xh^np RiiciiQthii<; directlv derived from the portion of the COM -BUbwnic^ 

the user, it ^ 'mporanx lo OR-BUS, whose construction appears to be novel in the art of 

circuite 11-1 to 11-N of the ZISC chip 10. thereby allowing a complete exchange of data therebetween that is totally 
independent of the ,3 to perform an OR function between all the local categories held 

. f™r/Fi?E orSoral is set to "l" this foroes global result signal UNC/FIRE.OK- to be in turn set to 1 . Signal 
nSFBF«-i?irJsL?rcfce(UNC/FlREOK-.1)whena^ 

^SC ohfp" «eScategorles assriclated therewith. In other wo* the input ve,»or has been reoog- 

"'^1h^';rnl^rrlr.tt"S^=;r.^:x^ 

drcuit (F- . 1) and without amblguit, (UNC/FIRE.OK- . 0) by neural network 1 1(A). The UNC/FIRE.OK signal 

the so-called Dmin determination process. In fact, the fictitious aggregate c.rcuit mentioned above .s constructed so 
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during the Dmin determination process, search/sort circuits are self-excluded until only the search/sort circuit of the 
neuron circuit(s) holding the minimum distance Dmin remains active. As a final result, only the neuron circuit which holds 
the minimum distance Dmin will be permitted to output the same to the OR circuit 12 via its NOUT-BUS. The value of 
the minimum distance Dmin (which is not modified in OR circuit 12) is finally available on the OR-BUS via the OUT*- 

5 BUS portion of the COM*-BUS. In the meantime of the said Dmin determination process, the local output signals NOUT 
that are outputted by the neuron circuits on their respective NOUT-BUS are not of significance. Similar reasoning applies 
to the determination of the minimum category Cmin among all the local categories CI to CN stored in the local category 
register 450 of each of said neuron circuits 11 -1 to 11 -N respectively. The minimum distance Dmin can be loaded in the 
AIF register 352 of the ready to learn neuron circuit via the OR-BUS, as the AIF value of this neuron circuit, during the 

w engagement process of the learning phase. 

The global result and output signals mentioned above are preferably applied to block 18 for storage therein. The 
role of block 18 will now be described in more details by reference to Fig. 6. Now turning to Fig. 6, block 18 Is connected 
to a block 22, which aims to represent the micro-controller or the user (referred to herelnbelow as the micro-controller/user 
22). via the SEL/CTL-BUS and the INDATA-BUS. The SEL/CTL-BUS is comprised of a selection bus SEL-BUS and a 

15 control bus CTL-BUS. In particular, the SEL-BUS transports the signals for appropriately selecting the various registers 
of the ZISC chip 10. The CTL-BUS conveys standard control and clock signals such as STROBE, CHIP SELECT , 
CLOCK, ... , and the like from the micro-controller/user 22. The function of the decode logic circuit 181 Is to generate 
said selection signals to allow access of a specific data in the adequate register. Depending on the value of these 
selection signals and control signals appearing on the SEUCTL-BUS. the control logic circuit 182 generates all the 

20 signals that are internally required for a proper operation of the ZISC chip 10 including the address signals that are 
required for accessing the R/W memory circuit 250 which stores the prototype vector components. Address and control 
signals are outputted from the control logic circuit 182 via a bus labelled lADD/CTL-BUS. The state machine circujt 183 
controls the sequencing of the ZISC chip 10 operation, and in particular, during the essential learning and recognition 
phases. Preferably, a status/cti circuit 1 84 is implemented, because it may reveal to be of interest for data exchange with 

25 the micro-controller/user 22. It is essentially comprised of a register 184.1 and a two-way AND gate 184.2 whose role 
will be explained In more details later on. As a matter of fact, register 1 84. 1 can store the global result and output signals 
that are generated by neural network 1 1 (A) as a whole and also control signals that are processed in the control logic 
circuit 182. The global result signals F* (or NID*). DEGOUT*, UNC/FIRE.OK* and ID* may be loaded into register 184.1 
of block 18 of ZISC chip 10. In particular, signal ID* is reconstructed from the F* and UNC/FIRE.OK* signals in AND 

30 gate 184.2 before being stored in a specific position of register 184.1. This position must be easily accessed by the 
micro-controller/user 22, because it contains the key information as to the input vector has been finally correctly identified 
or not by the neural network 1 1(A) incorporated in the ZISC chip 10. In particular, said specific position may be advan- 
tageously connected to a LED outside the ZISC chip 10 via a connecting wire for the Immediate and visual information 
of the user. These global result and output signals are transported on the M-BUS. In addition, register 184.1 can also 

35 store set-up parameters, data related to the operating modes of neural network 1 1 (A) for bi-dlrectlonal communication 
with micro-controller/user 22 via the INDATA-BUS. In particular, its contents may be accessed by the micro-controller/user 
22 during a WRITE or a READ operation. All circuits of block 18 are considered conventional to the man skilled in the 
art. However, the ZISC chip 10 of Fig. 4(A) is preferably provided with an interface circuit in logic circuit 182 that Is 
adapted to interface the various resources of ZISC chip 10, including registers, with the most commonly used standard 

40 buses such as the PCI, ISA and the like and their respective protocols. 

As described above, by reference to Fig. 4(A), the on-chip common communication bus COM*-BUS is formed by 
merging the R*-BUS and the OUT*-BUS. The COM*-BUS thus transports the totality of the response of the ZISC chip 
10 to presentation of an input vector. The OR signal is identical to the OUT* signal. The ZISC chip 10 of Fig. 4(A), thus 
incorporates neural network 1 1 (A) comprised of neural unit 1 1 (#). OR circuit 12, the COM*-BUS and the OR-BUS (the 

45 OR-BUS may be considered as a part derived of the COM*-BUS). The above described architecture allows a single 
ZISC chip 10 of Fig. 4(A) to operate in a stand-alone way for applications which do not require an external output bus. 
However, still according to the present invention, this described architecture can be easily modified to have the ZISC 
chip 10 of Fig. 4(A) provided with such an external output bus to operate in a multi-chip environment, as it will now be 
described hereafter by reference to the architecture variant of Fig. 4(B) still referenced 10. Likewise, the neural network 

50 incorporated in the ZISC chip 10 of Fig. 4(B) is also referenced 11(A). The aim being to produce a neural network of 
any desired neural capacity resulting from the combination of as many neural networks 1 1(A) as required by the appli- 
cation. This is accomplished by assembling the adequate number of ZISC chips 10 once adapted to operate in a multi- 
chip environment. 

To that end. should a plurality of ZISC chips 10 be assembled, the global composite result and output signals R* 
55 and OUT* generated by said plurality of ZISC chips 10 are ORed on another common communication bus included In 
the Inter-neuron communication system. This common communication bus is now necessarily located outside the ZISC 
chips 10 and will be referred to hereinbelow as the off-chip common communication bus. Preferably, this additional OR 
function is performed by adequate driver circuits directly on the said off-chip common communication bus. 
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M * Pin A(^\ thP RUS and the OUT*-BUS are applied to a block 19 which consists of standard driver. 

cSM-.ZtTleTon6 asterisk clearly indicates that an additional OR function has been performed on the R and 
ni IT* qinnals at the outDut of these driver circuits DR*. « ^ 

H^vt instead of being directly connected to its corresponding driver circuits DR*. the OUT'-BUS ca- be ojrt,on- 
ally core^Sd" th^input of a block 20. Basically, block 20 has the same construction as ^--^^^'^'Z'^^'^^^^^ 
Tearntron circuit 11 and will be referred to below as the main Dmin deterrn,na.on "^^"^ °* ^^^f^^^^^^^^ 
Uktw^se main Dmin determination circuit 20 is provided with a connection to a feedbackbus labelled OR -BUS_ln case 
roLdetermin^^^^^^^^^^ 

rnS:gt?s;:u'^^^^ °' ^'^C Chips 10. it may reveal useful. As a m 

of ac^ 72?imp;^^^ the overallprocessing speed of any neural network resulting of 

BUS is the essential constituent of a general purpose output data bus referred o ^f^^'-^^J.^ 

the implementation illustrated in Fig. 4(B). the GDATA-BUS may include signals 

The GOM-BUS may be connected to the micro-controller/user 22 either directly ^^rl ° S Zil^D chj 

Thanks to the OR function performed in the driver circuits DR* of block 19 on the COM -BUS, a plurality or ^'=>^^" 
{oma^S on?ectitreon.AsaresuU^ 

th«roofl rp iniected in the ZISC chip 10 via a driver/receiver circuit DR/RR of block 19 and the P-BUS. On the otne 
thereof) is both the OR-BUS and the M-BUS which is applied to the status and control 

21 (in this case the COM*-BUS is not formally required), and the neural network 1 1(A) incorporated ^uch as 

nli^ibp^ i^Fia 4mi win be referred to as the base ZISC chip and the base neural network respectively in the following 

he Fig.4(B) implementation. In the latter case, the signal transported by the OR-BUS ^^.T ^ ^"'^ ^„.„3te3 
0 In thecourse of the following description, the notations introduced above will be maintained. The ^VPhen designates 

that is incorporated in the ZISC chip 1 0 of Fig. 4(B) is referred to by reference 1 1 (A). 
In summary, at the neuron circuit 1 1 level, there are six input buses: 



so 



field 



55 



the 8-bit NO/CXT-BUS which conveys the 1 -bit input norm and 7-bit input context data; 
Still at the neuron circuit level, they are two output buses: 
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the NR-BUS which transports the composite local (or neuron) result signal comprised of signals F, DEGOUT and 
UNC/FIRE.OK. Signal F is illustrative of the "fire" response of a committed neuron circuit to the input vector pres- 
entation, i.e. if it has been recognized or not. Signal DEGOUT is illustrative of the "degenerate" status, i.e. if neuron 
circuit that has fired is degenerated or not. Signal UNC/FIRE.OK results from the comparison between the local 
5 category C of neuron circuit 1 1 with the global category C*. This signal aims to remove the ambiguity as to the 

category of an input vector that has been recognized; and. 

the NOUT-BUS transports the local (or neuron) output signal NOUT. In either mode, at the end of the Dmin deter- 
mination process. NOUT = Dmin for the neuron circuit which holds the minimum distance Dmin and the NOUT 
signals of the other neuron circuits are not of significance but have no influence at the OR circuit 12 level. When the 
10 Dmin determination circuit 500 is passed-through (more exactly, it does not process the signal applied to its input), 
the NOUT signal may represent the value held in any register or memory of neuron circuit 1 1 . Usually, the NOUT- 
BUS transports the distance calculated by the neuron circuit or the local category held in its category register 450. 
However, in the SAVE and RESTORE modes, it transports other data (set-up parameters, prototype vector compo- 
nents and the like). 

15 

At the ZISC chip 10 level, there is no distinction between either implementation as far as the Input buses that interface 
the chip with the micro-contro!ler/user 22 are concerned. These input buses are : 

the 16-bit bi-directional INDATA-BUS which transports the input vector data, the set-up parameters, the Input 
20 norm/context and category data to allow a complete exchange of data between the micro-controller/user 22 and 
the ZISC chip 10; and, 

the SEL7CTL-BUS which similarly transports the selection and control signals to the ZISC chip 10. / : 

Now. there is a distinction as to the output bus at the ZISC chip 10 level. With the Fig. 4(A) implementation, the 

25 output bus is the on-chip common communication bus COM*-BUS which results of the R*-BUS and OUT*-BUS combi- 
nation after generation of their respective signals in OR circuit 12. The R*-BUS transports the global result signals F*. 
DEGOUT* and UNC/FIRE.OK* that are illustrative of the global response of the neural network 1 1 (A) to the presentation 
of an input vector A. The OUT*-BUS transports the global output signal OUT* which essentially represents either the 
minimum distance Dmin (or Cmin) among all the distances (categories) calculated (held) by the neuron circuits of neural 

30 network 1 1(A) at the end of the Dmin (Cmin) determination process or the global category C*. 

The Fig. 4(B) implementation is adapted to a multi-chip environment. As a result, beyond the driver, receiver and 
driver/receiver circuits of block 19. the off -chip common communication bus COM**-BUS is now required to interface 
the base ZISC chip 10 with the external world including the other base ZISC chips 10 that are to be connected thereon. 
The COM**-BUS results of the R**-BUS and OUT**-BUS combination. The R**-BUS transports the global result signal 

35 F**. DEGOUT** and UNC/FI RE.OK**. The OUT**-BUS transports the global output signal OUT**. The OUT**-BUS feeds 
the OR-BUS of each neuron circuit 11 (assuming no main Dmin determination circuit 20 is provided in the base ZISC 
chip 10). A GDATA-BUS including the 17-blt COM**-BUS may be implemented if other wires are required otherwise the 
COM**-BUS will suffice for base ZISC chip 10 interconnection. 

Moreover, at whichever the level is, the connection between all the neuron circuits of either implementation requires 

40 a serial daisy chain connection, i.e. the two wires which transport the daisy chain input and output signals (DCI. DCO). 

* VARIANTS OF THE EiASE ZISC CHIP ARCHITECTURE * 

With regard to the architecture of the base ZISC chip 1 0 illustrated in Fig. 4(B). there are several architecture variants 
45 that may be envisioned. First of all, the various OR functions performed in OR circuit 12 may be easily distributed, and 
thus incorporated in each neuron circuit 1 1 of base ZISC chip 10. In this case, as illustrated by reference to ZISC chip 
10* in Fig. 9. the common communication bus has now a serial structure comprised of bus elements referenced COIVI**- 
1-BUS to COM**-N-BUS. The GDATA-BUS still remains optional. The solution is relatively simple, but the full parallel 
structure of the base ZISC chip 1 0 of Fig. 4(B) is no longer true, and this solution results in a significant degradation of 
50 the overall processing time during the learning and recognition phases. The COM**-1-BUS of the first ZISC chip 10' in 
case several chips are cascaded in a chain are connected to a logic level that is neutral for the OR function (equal to 
zero as mentioned above). 

Another valuable variant derived from the Fig. 4(B) implementation can be designed as an alternative. OR circuit 
12 may be physically suppressed. The OR function performed In OR circuit 12 which Is necessary to the operation of 
55 the base ZISC chip 10 could be achieved directly by OR dotting of the NR and NOUT signals directly on the COM**- 
BUS via a driver circuit of the DR* type for each neuron circuit. Moreover, it could be also desirable to further distribute 
and incorporate all the functions performed by blocks 13 to 18 in each neuron circuit 1 1 . In this case, illustrated in Fig. 
10. the ZISC chip 10" would be just limited to a plurality of neuron circuits labelled 11 "-1 to 1 1"-N forming neural unit 
1 1 '*{#). Such a neuron circuit 11" could be introduced in an ASIC library and lead to a growable neural network macro. 
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Since it would have a granularity Of 1 (instead Of 36 in me part^c^^^^^ 

and 10- Of Fig. 9 described above). In ^^J^ 'TPj^^T^JS^^^^^ In the solution 

interconnects a deterrr.m«l neuron c.cu.t 1 1 .^'^^^^^^^ buU«,uiri as many times thirty one (3.14. 14) wires 
of Fig. 10, the structure of the ZISC chip 10 is *""y Pfj" q ^^^^ s,de of the ZISC chip 10". the two 

;r the Smpa'^nli a^^^ g^^n^^of the design of ZISC chip 10". Still further va.ants inc^d^ intermediate arch,- 
tSureiZ be designed. However, the architecture shown in Fig. 4(B) is preferred for its versatility. 

* THE ZISC MODULE ARCHITECTURES * 

TO extend the neural capacity, an elementary ZISC --^"'^ ^ 'ND^fBuTand sT^^^^^^^^ 

end, a specBic terminal labelled INH of all ZISC chips 10 ^^^^^ is cijineeted to a second 

, say ,« suppl, voljago t",^';, .e.n*na, o. the DR/RR 

potential, in this instance the ground GND In fact this '1^" ^® . ^^ appHed to the said 

circuit which interfaces the ZISC chip 10 with the bidirectional 'N°ATA-BUS^ W^^^^ s W 

DR/RR Circuit, the latter is driver^n a JiQ/^^^-p^^-^^^^^ ^oVrBU^wS isth'sTom^^n to all the ZISC chips 

s roTi)1o~=^^^^^ 

thereof for dotting thereorr gDATA-BUS inter-connect all the chips, an elementary ZISC module 

Because, the '^DATA-BUS SELJCTL BUb ana ou ^ ^ ^^^^ ^^^^^ 

nssr Sc^'rnSrsr r:=orr OR do». ,^e 00.™*^ 

complex module 10{A} by assembling two e'«'^^"'«^y,'::°^"'«V*?r;lnSioTi the DcTand ^NH e^^^ of the 

;i-ro-Ve7ii7rrs;^o^^^^^^^^ 

rjret n^rot^^roS ra"StLant\.uc.. .depan.. on .e 
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capabilities of the technology), so that a complex neural network now including for instance, hundreds of thousands 
individual neuron circuits is obtained. AH these neuron circuits are working as if they were integrated in a single base 
2ISC chip 10 and their number is transparent for the user. 

In the course of the following description, the notations introduced above will be continued on the same basis. To 

5 distinguish between different ZISC chips in the assembly that forms the elementary ZISC module 10[A], as mentioned 
above, parenthesis have been adopted for chip related reference (e.g. chip (q) designates the qth chip in the chain). 
Square brackets designate a reference related to the elementary module. Likewise, the buses and circuitry associated 
to this elementary module 10(A] will be labelled the same way. For instance, to distinguish between different ZISC ele- 
mentary modules that are assembled to form a complex module 10{A}. reference 10[r] designates the rth elementary 

10 module in the assembly. Finally, the buses and circuitry associated to this complex module will be labelled the same 
way. The complex neural network which is thus incorporated in any complex module 10{A} is referenced 1 1{A}. 

In the above description, reference has been made to successive OR functions. These OR functions may be per- 
formed either by hardware (e.g. OR circuit 12) or by dotting (e.g. the hard-wired OR function on the COM**-BUS). 
On the other hand, these OR functions may be achieved through AND functions according to the de Morgan laws. 

15 should it be so required by the technology. For instance, the global result fire signal F* has been defined as resulting 
from the ORing (in a dedicated OR sub-circuit) of all the local result fire signals of the neuron circuits 1 1 -1 to 1 1 -N. i.e. 
F* = F1 OR ... OR FN. Depending on the technology used, implementing the OR dotting may reveal to be present some 
inconveniences, because it requires PFETs that are known to be slow devices. The objective being to determine if at 
least one neuron circuit of the neural network has fired, i.e. if at least one signal Fi is equal to 1 , it goes without saying 

20 that the same result could be achieved if at least one H= 0. In this case the above relation is equivalent to F"" = FT AND 
... AND FN. The AND dotting which only requires NFETs that are admittedly fast switching devices is therefore much 
more popular. s ; 

BRIEF DESCRIPTION OF THE BASE ZISC CHIP OPERATION 

25 

A brief description of the essential operating modes of the neural network 11 (A) formed in the base ZISC chip 10 
of Fig. 4(A) will now be given in conjunction with Figs. 13 to 15, which schematically illustrate the flow charts of the 
INITIALIZATION. RECOGNITION AND PRE-CHARGE and LEARNING modes respectively To facilitate reader's under- 
standing some references are made to the functional blocks described in Figs. 4 to 12. 

30 

* INITIALIZATION MODE * 

As apparent from the flow chart referenced 24 in Fig. 13. the INITIALIZATION mode essentially consists of one step 
(box 25) referred to as the clearing step. During the clearing step, the contents of certain registers of the base ZlSGxhip 

35 10 (including those of neuron circuits incorporated therein) are loaded with adequate default values. MaxlF register 14 
and MinlF register 15 are loaded with respective MaxlF and MinlF values. A predefined (e.g. zero) value is loaded in 
main NO/CXT register 17. A zero value is loaded in the DAISY register of daisy chain circuit 600 and in the category 
register 450 of each neuron circuit 1 1 . 

Other registers and the R/W memory circuit 250 do not require to be loaded with predetermined values, their respec- 

40 tive contents are thus undefined at power-on. However, a second step of updating (box 26) may be envisioned. In this 
optional step, the micro-controller/user 22 may optionally change some values, e.g. the set-up parameters (MaxlF, MinlF), 
the input context and norm data loaded in main NO/CXT register 17 and the content of register 184.1 which in particular 
holds operating mode related data in addition to the global response related data. These steps are achieved via the 
INDATA-BUS under the supervision of control signals transported on the SEL/CTL-BUS. 

45 

* RECOGNITION AND PRE-CHARGE MODES * 

Let us now consider the neural network 1 1(A) in normal operating conditions after the initialization phase has been 
completed as described above. Let us assume that certain neuron circuits have been already engaged (i.e. they have 

50 learned), while other still remain free. The micro-controller/user 22 successively loads the components of an input vector 
A in the A register 13 of ZISC chip 10 via the INDATA-BUS. To that end, the A register 13 is selected via the lADD/CTL- 
BUS. The data loaded in the A register 13 are then transmitted to each of the neuron circuits of neural unit 1 1(#). i.e. 
those that have been engaged as well as those still free. This distinction between free and engaged neuron circuits 
results of the consideration of the logic levels of signals at the respective DCI and DCO terminals of daisy chain circuit 

55 600 for each neuron circuit 1 1 . Fig. 1 4 is comprised of Figs. 1 4(A) and 1 4(B). It represents the flow-chart referenced 27 
which illustrates the essential steps that are performed in the ZISC chip 10 during the recognition phase. Now turning 
to Fig. 14(A). it is apparent that only a neuron circuit 1 1 that has been engaged (box 28) and selected by the NS signal 
(box 29), i.e. only a committed neuron circuit, will participate to the recognition phase in a significant manner. An neuron 
circuit is engaged as soon as it has learned, i.e. it is no longer free. A neuron circuit is selected when the local context 



BNSDOCID <EP 0694855A1> 



19 



EP 0 694 855 A1 



10 



15 



20 



25 1 



30 



portion (Cxt) of its norm/context register 100 matches the input context portion (CXT) held ^'^^^'^^^ 
^y. However, if said input context portion is equal to zero, all the engaged 

eration of the qlobal result and output data. r. l. ♦u^ 

Afte he Lt input vector component has been applied by micro-controller/user 22, the d'Jance D be^.een the 

the test conducted in box 34 is to determine -J^^^ ° J^^^^^^^ Fo ^a ietern^^^^^^^^^^ 

AlF). neuron circuit 1 1 does not f ,re and signal F s set .nacw^ e. F 0^ in box^SS- Fo^ ^^^^^ .^^^ 

the recognition processus is terminated. If YES. i.e. D < AlP. signal li is sei -s^"™ v aoDlied to the 

vector A falls wShin the AlF value of the prototype vector stored in neuron circuit ^l^^ ^nal LT wh^^^ he 

mSFIRE OK - 0) so that global result signal UNC/FIRE.OK* is equal to zero. Consequently, signal 'D = F AND 
fSrS-DK' rs setSt ve i e ID* = 1 (box 42). The input vector has been identified with a category equal to C . The 
^2,^rv CMS tl^^rr^n^ifable on the COM*-BUS and can be stored in register 1 84.1 . The category may be read 

brthel-^^^^^^^^^^^ 

;oxef43ar4TarrSr rnacommonnode^a^ 

controller/user 22. either the distances or the categories may be made available °" ^^^^.^OJ^^;,^^^^^^^^ 

Sjtf r/Sn one' ne'uron circuit has fired, the distances and the -'ated ca^^ories can J^^^Pf^^^^^^^^ 
continuous sequence or in an interleave fashion. If only one neuron circuit has fired, only its distance or its category 
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data are outputted on the OUT*-BUS. The distance or the category data are sent either directly to the micro-control- 
ler/user 22 or latched in register 1 84. 1 via the COM*-BUS for a subsequent READ operation by the micro-controller/user 
22. 

In the nearest neighbor sub-mode (NNM), there is no comparison between calculated distance D and the AIF value. 
5 In this case, the micro-controller/user 22 only requires all the search/sort circuits of all neuron circuits without any restric- 
tion (having fired or not) to determine the minimum distance Dmin (or Cmin) and then to output the remaining distances 
(or categories) just as mentioned above. As apparent from Fig. 14, the NNM sub-mode only comprises the steps per- 
formed in box 46. 

Finally, irrespective the selected sub-mode, the RECOGNITION mode ends in box labelled END in Fig. 14(B). How- 
10 ever, because at the end of the recognition phase illustrated by reference to Fig. 14, a neuron circuit 1 1 may be then 
submitted to a learning phase, a box 47 has been added as an alternative to the termination box mentioned above. 

* LEARNING MODE * 

15 At the end of the recognition phase, the user may carry out the LEARNING mode. Learning occurs in different cases. 

For example, if the user is not satisfied by the global response, e.g. the category outputted by the neural network 1 1(A) 
is not deemed correct or when a "blank" or "virgin" neural network 1 1 (A) is used for the first time. In the latter case, none 
neuron circuit having learned, none can fire so that signal F is equal to 0 for each neuron circuit as it has been made 
apparent from box 35 in Fig. 14(A). As a result, signal F* = 0, signal UNC/FIRE,OK* = 0 so that ID* = 0, which means 

20 the input vector A has not been identified at the neural network 11 (A) level. According to a significant feature of the 
present invention, during the previous recognition phase, the components of the input vector A have been stored in the 
first free ready to learn neuron circuit which is thus pre-charged as explained above by reference to box 31 in Fig. 1 4(A). 

Let us still assume that certain neuron circuits have been already engaged (i.e. they have learned), while othierlstill 
remain free. The LEARNING mode basically consists in the reduction and/or the engagement processes, which are 

25 directed to neuron circuits having a different status. The reduction process concerns all the committed neuron circuits 
while the engagement process concerns only the ready to learn neuron circuit. Let us assume for sake of simplicity that 
the learning phase is conducted after the recognition phase as described by reference to Fig. 1 4. The flow-chart labelled 
47 in Fig. 15 for consistency with box 47 of Fig. 1 4, aims to illustrate the essential steps of these reduction and engage- 
ment processes. The learning phase is performed under the supervision of the state machine 183 of the status and 

30 control logic circuit 1 8. The state machine 1 83 provides states in sequence to control all the neuron circuits of the neural 
network 1 1(A) in parallel. For any committed neuron circuit that has fired during the recognition phase (see flow-chart 
27 of Fig. 14) and has a local category C different from the input category CAT. there is proceeded to an automatic 
reduction of its influence field value. To that end, the ZISC chip 10 is provided with an adjustment mechanism which 
automatically and simultaneously reduces the contents of the AIF register 352 of that determined neuron circuit that has 

35 wrongly fired to remove the input vector from the firing range thereof. On the other hand, it may be worthwhile if the input 
vector has not been correctly recognized at the neural network 1 1 (A) level to engage a new neuron circuit to be selected 
among the free neuron circuits. These two cases will be separately discussed in more details by reference to Fig. 15. 

Now turning to the left part of Fig. 15 which illustrates the case of a committed neuron circuit that has fired, the 
learning process is initiated in the box labelled START Such a neuron circuit 1 1 is first tested in box 48 to determine 

40 whether the input category CAT and local category C of neuron circuit 1 1 are equal or not. If CAT and C are different, 
this means neuron circuit 1 1 has wrongly fired. This also includes the case where CAT = 0, because the local category 
of a committed neuron circuit is forcedly different of zero. In this case, the UNC/FIRE.OK signal outputted by identification 
circuit 400 of that determined neuron circuit is equal to zero. The meaning of the UNC/FIRE.OK signal in the LEARNING 
mode is to indicate that a reduction process is necessary The reduction process of said neuron circuit 1 1 is required to 

45 decrease its AIF value. In addition, the calculated distance D stored in a dedicated register of evaluator circuit 200 is 
loaded in the AIF register 352 (box 49). Now distance D is compared with MinlF in compare circuit 300 to generate the 
LTE signal. According to box 50, if D is lower or equal to MinlF, neuron circuit 1 1 is declared degenerated, i.e. signal 
DEG generated in the identification circuit 400 is set active (DEG = 1). Practically, a "one" is loaded in the DEG register 
(labelled DEG REG.) as illustrated by box 51. The DEG signal functions as a control signal to have the MinlF value 

50 loaded in said AIF register 352, according to box 52, ovenwriting thereby the previous D value. The NO output of box 50 
and the output of box 52 are merged in common node 53. Now turning again to box 48, if CAT = C, this means that the 
neuron circuit in consideration has rightly fired and therefore its AIF does not need any reduction. The UNC/FIRE.OK 
signal of that determined neuron circuit 1 1 is set active (UNC/FIRE.OK = 1) as illustrated in box 54. The output of box 
54 is connected to said common node 53. Finally in box 55, the minimum distance Dmin and the value of global result 

55 UNC/FIRE.OK* are determined. Both values are required in the engagement process of the ready to learn neuron circuit 
(if any). They are made available on their respective buses as described above by reference to Fig. 14, e.g. for being 
latched in register 184.1. 

Now, let us consider the right part of Fig. 1 5 which illustrates the engagement process for the ready to learn neuron 
circuit 1 1 . In this case, the process initiated in the box labelled START directly corresponds to the box labelled END in 
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Fig 1 4(A), i.e. we are still at the neuron circuit level. First, the value of the input category CAT is t^=;«lj;t«x 56^ Le^ 
whether CAT is equal to zero or not. H CAT = 0. the engagement process stops, (it is only required to proceed to the 
:^:So.7^S"^\^e cLm^ed neuron circuits as mentioned f °- On *he com^^^^^^^ 

the minimum distance Dmin (whose value has been established m box 55) .s '°^ded t^^^^^^^^^ Sf s iSded nste^^^ 
to learn neuron circuit according to box 57. Note however that, rf Dm,n ,s greater than Max F. Maxlf s loaded .nsteaa 
N^vt a comoarison is oerformed in box 58 to determine whether Dmin is inferior or equal to MinlF. If YES, a one is 
^o^L in trOEG regi^^^^^^ circuit 400 and the MinlF value is loaded in the AIF register 352 (box 59), 

'Se StLut 0^ box 59 and the NO output of box 58 are merged in common node 60. Now, the value of 9'obal result signa 
UNG^fSe SK°also deiermined in L 55 at the neural network level is tested. Through this test, .t is checked (box 61 
wSer the L^^^^^^^^^^^ recognized by at least one neuron circuit with the right category o-" "ot; » .^"f ' 

uSfirE/OK* - 1 th°s means the input vector has been correctly recognized by at least one neuron circuit with the 

rightSre^STlnViscTse if^^^^^^ 

iTos^ Ss^ageof theengagementprocess^ 

foUhe engaSment proces^ continues. The input category data CAT are loaded in the -tegory reg^^^^^ 
CAT BUS and the input norm/context data are loaded in the main NO/CXT register 1 7 via tl^^NO/CXT-BUS d the ^Ki 
feady to learn neuron circuit (box 62). Finally, signal ST is activated to shift the "one' va ue -^'iJ^^^^Mh J^^^^^^ 
inout terminal of the daisy chain circuit 600 of the ready to learn neuron circuit into the DAISY register thereo' ^s 
KrSby b^x 63 The engagement process is now terminated, the ready to learn neuron circuit has Just learned 
?he S5 c^a^input and ou^uJsignals of this newly engaged neuron circuit are equal to one. lejnmg ph^^^^^^^^^ 
when the AIF value of all the committed neuron circuits that have wrongly fired have been readjusted and/or the newly 
engagS nrori circuit has its AIF value established, its local category and norm/context registers properly loaded. 
Alternatively, the test completed in box 61 may be performed between boxes 56 and 57. 

At the very beginning after initialization, when none neuron circuit of neural network 1 1(A) has learned, a^^neuron 
circuVte are f^ee During L recognrtion of the first input vector ever applied to neural network \l(AMhe cor^ponents 
S^Jreof a e succes^S applied to the RAW memory circuit 250 of each of the neuron circuits of ZISC chip 10 via the 
S?A Bul Of course Sie calculation that is performed in the distance evaluator circuit 200 and the comparison that 
o n^S m^in th?di?tJnce circuit 300 of any neuron circuit are not significant. Because the first input vector 

rial resuime signal F which, which is ORed with other F signals in OR circurt 12 to produve a global ^esul f-re signal 
^Sh ts eauatto zero (F* = 0 The ready to learn neuron circuit, i.e. 1 1 -1 , therefore needs to learn this input vector, as 
Thas beS exS above But. as a Lult of the recognition phase, the components of this first input vector have 
Len pre ou7y stoSi and this first (free) neuron circuit is now pre-charged. It iust f ^-^J? ^^^^^^ 

neuron circurt, as described above by reference to Fig. 15. to have the first input vector stored as the first prototype 
vector, in this case, note that the MaxlF value is loaded in the AIF register 352. 

Therefore an efficient way of precharging the components of the input vector A in the R/W memory circuit 250 
during th:?:^gni^^^^^^^^^ has be'en im^em'ented in the ZISC chip lO^MoreoveMhe r^^^^^ 'Z'^SS'SZT::. 
easily recognized, because it is the only neuron circuit of the neural network 11(A) whose and DCO signals a e 
comolememarv o each other Unlike the neural networks of the prior art which require the operation of storing the nput 
vSo com^nel^ during a preliminary step of the learning phase (during which the '"Put vectons 

^^L^Zi^Z to the neural network) in the ZISC chip 10. the said components have been previously loaded in the 
?^ r^elrfc^Jcu^^^ Phase o?a input vector. As a result, the ready to learn neuron circuit is 

Zr^atSl J aTmrr^L^ypre-oUarged. i.e. prepared for being subsequently engaged if so required by the neural 

et^^rkS^ 

loaded in category register 450 of neuron circuit 1 1 and signal ST set active to load a one in the DAISY register or 
. SSsfd^iai^S 600 in all cases where the learning is not required, the stored components will be o-^"*^; 
fhTcompon nts of the next input vector are presented on the INDATA-BUS in a f^,^ 
as this neuron circuit has been engaged, the control signal RS prevents any subsequent WRITE operation in the R/W 

"^TIu" rSse ZISC chip 10, once the input vector to be learned has -^^J^^J^^^^i 
7 Circuit 250 of the ready to learn neuron circuit (as described above with regard to the PRE-CHARGE -^ode tt^at s 
a compSheL during th'e recognition phase), the learning mode includes the -^age-ent o 
npuron circuit The enqaqement process terminates when the desired input category is loaded into the category register 
JsS andTone^s loaS '^e DAISY register of the daisy chain 600. The learning phase also includes the reducjon 
pf^Ss Whenever required to determine the new value that must be loaded in the AIF register 352 o^th-^^^^^^^^^ 
5 neuron circuits that have wrongly fired. All the related steps of the learning phase are conducted by the base neural 
neZk T( W Susrmerts occur simultaneously and automatically in all neuron circuits of the neural network and 
do not ^il^re fny e^ma supervision, for instance by the micro-controller/user 22. Therefore, the learning processus 
fs fX S and automatic. The state machine 183 sequences the reduction and the engagement Processes of all 
hl^eu^n clits that are concerned in parallel. In addition, contrarily to conventional neural networks, and in accord- 
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ance to a key feature of the present invention, the phases of recognition and learning are not totally distinct because of 
the pre-charge feature. 

* SAVE AND RESTORE MODES * 

5 

Finally, the base ZISC chip 10 offers two other operating modes that are worthwhile: the SAVE and RESTORE 
modes. Because the present implementation of the ZISC chip 10 do not include EPROMs. when power is shut-off. all 
the information held in the different registers of the ZISC chip 10 are lost. As a consequence, the SAVE and RESTORE 
modes are highly desired. During the save phase, all the valuable data that are stored in the various registers of the 

10 base ZISC chip 1 0 (including those of neuron circuits) may be accessed (READ operation) by the micro-controller/user 
22, then saved. For instance, this concerns the components (weights) of the prototype vector B stored In R/W memory 
circuit 250, the AIF value stored in the AIF register 352, the local category stored in category register 450, the local 
norm/context data stored in register 100, the set up parameters, ... etc. A hard-disk is a convenient storage means in 
that respect to appropriately store these data. The RESTORE mode-basically consists of the reverse steps. For different 

15 reasons, the micro-controller/user 22 may wish to load either said data in their respective registers for restoration thereof 
(WRITE operation). During the restore phase, it is important that any neuron circuit be loaded either with its own data 
or with any desired data as the case may be. Obviously, it must exist a perfect correspondence between the data and 
the neuron circuit registers. With the present implementation of the ZISC chip 10. the SAVE and 'RESTORE modes 
advantageously take benefit of the presence of multiplexer 501 in the Dmin determination circuit 5pb (Fig. 8) and of the 

20 inherent characteristics of the daisy chain circuit 600. 

Let us first consider the SAVE mode. Now turning again to Fig. 8, multiplexer 501 is adequately activated during 
the save phase to select the desired input bus (and thus the data) among its three input buses. Depending on the 
selection signals that are applied to multiplexer 501 via the lADD/CTL-BUS, either the local category data C, theliocal 
norm/context No/Cxt data or the data appearing on the J-BUS are applied as a first input signal to search/sort circuit 

25 502. Depending on the selection signals that are applied to multiplexer 351 via the lADD/CTL-BUS (see Fig. 7). in turn, 
the J signal may represent the contents of the AIF register 352 or the data appearing on the DIST-BUS of the neuron 
circuit 11 in consideration or the MaxIF/MinlF value transported on the MaxIF/MinlF-BUS. The daisy chain circuit 600 
is used to select the appropriate neuron circuit. The same principles described above with first free neuron circuit are 
also applicable here. To have the prototype vector B components successively appearing on the D IST-BUS, it just suffices 

30 to present an input vector A whose all components are equal to zero (see Fig. 5). Finally all these data pertaining to 
any determined neuron circuit are sequentially outputted from the Dmin determination circuit 500 of that determined 
neuron circuit on the OUT*-BUS via its NOUT-BUS and OR circuit 12 (all the other NOUT signals are set at a neutral 
logic level). 

In summary, for the SAVE mode, i.e. the data to be saved are read by the micro-controller/user 22 by sequeritially 

35 selecting the adequate data via control signals to be finally outputted on the COM*-BUS. then they can be stored in 
register 184-1 (see Fig. 6) via the M-BUS or directly made available on the INDATA-BUS via a multiplexer (not.shown) 
as the case may be. The data are outputted on the INDATA-BUS at micro-controller/user 22 request according to an 
interface circuit located in logic circuit 182 under the supervision of the state machine 183. The daisy chain circuit 600 
ensures that only one neuron circuit is selected at a time. 

40 For the RESTORE mode, the data are loaded by the micro-controller/user 22 via the INDATA-BUS. Each register 

is successively selected via the lADD/CTL-BUS to have the data properly stored therein. Same reasoning applies to the 
R/W memory circuit 250 of each neuron circuit. 

The SAVE or the RESTORE mode requires a few steps. First, the micro-controller/user 22 sets the adequate (SAVE 
or RESTORE) mode. In either mode, there is a preliminary step wherein all the DAISY registers of the ZISC chip 10 are 

45 reset. The micro-controller/user 22 selects the correct data corresponding to the correct register or the R/W memory 
circuit 250 by sending the adequate address on the SEUCTL-BUS. Neuron circuits are processed in sequence under 
the supervision of the daisy chain circuits 600 according to the same mechanism as described above in the engagement 
process by reference to Fig. 1 5. This step is repeated for each register as many times as necessary. The neuron circuits 
are read or written in sequence until the first neuron circuit whose local category value is equal to zero. The contents of 

50 the local category register 450 is thus a convenient means to detect the end of either mode. At the end of the save or 
restore phase, all registers of the neuron circuits, including the DAISY registers of daisy chain circuits 600, are in the 
initial state, i.e. in the same state that they were before the save or restore phase was initiated. Finally when all the 
registers concerned have been read or written, the last step consists to return in either the RECOGN ITION or LEARN I NG 
mode. The present approach needs a very few circuitry, because the same processing steps are used in either mode 

55 to access a specific register of the neuron circuits. In addition, there is no limitation in the number of neuron circuits 
submitted to either the SAVE or RESTORE mode. 

Similar reasoning applies to the base neural network 1 1(A) of Fig. 4(B) and can be further extended to any neural 
network incorporated in either the elementary module of Fig. 11 or the complex module of Fig. 12. This is a direct 
consequence of the fully parallel architecture of the base ZISC chip 10. The first global OR function is performed by 
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ORing the local result and output signals in respective dedicated sub-circuits of OR circuit 2 to Produce the Ql^al result 
R- and output OUT* signals for each of the ZISC chips 10(1) to 10(Q). In turn, these global s^nals that are labeHed 
R*(1 Ho rTq) and OUT*(1 ) to OUr(Q) respectively in Fig. 1 1 . are 0R«^ 
alobaignate 

tSw^Slcori.r^tes the complex neural network 1 1[A] as apparent from Fig. 11. In the ^^^J^^^'^^^^;^^^^^ 
ZISC chips 10 are assembled together, the main Dmin determination crcuit 20 optionally provided in each ZISC ch p 
10 according to the implementation of Fig. 4(B). may be advantageously used to determine '"'7™7^ff "^^^T, 
between the committed neuron circuits belonging to the all the chips forming the elementary module or a conplex 
rn^ule 10{A} As a result of neuron circuit and the chip cascadability which results therefrom, operation of any neural 
:eCk oSer the type 11[A] illustrated in Fig. 11 or Of the 

to the operation of neural network 1 1(A) of Fig. 4(A) described above by reference to Figs^ 13 to 15. ^I^e c^stinrtion 
beLeen a base neural network, an elementary neural network and a complex neural network is 
coSer/user 22. In other words, the number of ZISC chips 10 (each incorporating a base neural network) that are 
assembled to form an elementary module, which in turn, are assembled to form a complex module finally results in a 
SmpTex neural network of the delired size whose the total number of neuron circuits is ^""^ ^Pt^^H^iS t Lp^^ 
controller/user 22 Moreover, the operation of such a neural network is independent of its size. The different types of 
neuraVnetlloTks m^ above will be therefore referred to hereinbelow by the generic term of neural networks wher- 

ever the distinction therebetween is not necessary 

DETAILED DESCRIPTION OF THE BASE ZISC CHIP 

* THE CONTEXT REGISTER AND MATCHING CIRCUIT (100/150) * 

Fig 1 6 illustrates the detailed construction of matching circuit 150. whose role is to select or not the neuron circuit 
1 1 Matching circuit 150 first comprises NOR circuit 151 which checks whether the input context ^XT value is nul o^ 
not If input context CXT value stored in main NO/CXT register 1 7 is equal to zero, a logic one is generated and apphed 
tothefIinputofa2-wayORgate152. According to Fig. 16. thereisaNORclrcuitlSI ine^^^^^^^^ 
a single NOR circuit common to the whole base ZISC chip 10 could be envisioned as well (as the single "^am NO/CXT 
TegisL 17 which is common to all neuron circuits of ZISC chip 10). NOR circuit 151 ^^^^^l^^S^^'-^^l^^^ 
Matching circuit 150 further comprises a XOR circuit 153 which compares the input context CXT and ^^^^^^^^ 
Cxt data XOR circuit 153 basically consists of seven 2-way XOR gates. The signal which is outputted from XOR c^curt 
TS iriplil to NOR circuit 154. The latter basically consists of one 7-way NOR gate. The signal generated by NOR 
crcuit raS>li^ to the second input of OR gate 1 52. The signal generated at the output of OR gate 1 52 is the NS 
meuK^n Select^n) signal mentioned above. When signal NS is active (NS = 1 ). neuron circuit 11 is se^cted. In summary 
a neuron circuit 1 1 is selected only if the input context is null or if it does match with the local ^"^^^^^ 

Fig 1 7 illustrates the concept at the base of the context approach adopted in the disclosed implementation of the 
ZISC chip 1 0. All the neuron circuits 1 1 -1 to 1 1 -N operate in parallel on the same '"P"* ^°"text da^a CXT stor^^^^^ 
context portion of main NO/CXT register17.Thelocal context portioninNo/CxtregisterlOO IS loaded from mainNO/C^^ 

register 17 via the CXT-BUS during the engagement process of a determined neuron circuit, e.g. neuron circurt 1 1 Jhe 
context mechanism allows the dynamic selection of a determined number of neuron circuits forming base neural network 
1i?2Tora specif ic task. For instance, a typical example of application of the context consists in the selective recognition 
of a series of input vectors by a neural network by two groups of neuron circuits. The first group that has been engaged 
with the LI norm receives a first context (contextl) and the second group that has been engaged ^^^^ 
receives a second context (cont6xt2). Should it be required to proceed to the recognition of an input sector only by those 
o' neuron circuits that have been engaged with the LI norm, it is easy, thanks to the context ^-fferentia^on^to selec^ 
those of the first group and to exclude those of the second group. Another typical example can be found when some 
heuron c rcuits have learned with input vector of a first length (e.g. n = 64) and other with input ° ^/^^^^^^^^ 

length (e g n = 32). When input vectors of the second group are presented to the base neural network 1 1 (A), the neuron 
circuits of the first group must be inhibited, because their response would not be significant. 

iS^ever 7e above described concept may be generalized. Let us assume each neuron circuit 1 1 corresponds to 
a base processing unit. Now let consider a multi-processors unit built-up with a plurality of such base processing units 
handling an input data pattern in parallel as base neural network 11(A) does. In some casesjt may be requ|r«i to 
Srspatch the input data pattern only to one or a few base processing units. The present context approach avoids the 
?eceSy to design the pattern dispatcher that would have been normally required in front of the "^"'t'-P^^f ""J' 
. Such conventional dispatchers are known to be of great circuit complexity. According to the present ^^o^J^^^P^^f^^' 
this dispatcher would be reduced to a single main context register (which would correspond to the mam NO/CXT register 
iTof Rg 1 7) ln°urn. each base processing unit would then be locally provided with a dedicated local context regis er 
(which would correspond to the local No/Cxt register 100) and a bus that would interconnect the mam context register 
to the local context register of all the base processing units forming the multi-processors unit; Consequently, with such 
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implementation, before an input data pattern is presented to the multi-processors unit to be processed in one (or more) 
determined base processing unit, the input data pattern is preceded by an input context data which is directly loaded 
into the main context register. If the contents of the local context register matches with the input context value stored in 
the main context register, then the said input data pattern (and the following as well, provided there is no change in the 

5 context value) is processed by the said determined processing unit. An additional specific circuit may select at! the base 
processing units as soon as the input context is equal to zero. The penalty introduced by the above context feature in 
terms of delay in the data processing is reduced to a strict minimum. Obviously, there is no penalty if the context feature 
is not used (in this case the main context register contents must be equal to zero). There is a penalty to insert a context 
value within the input data pattern as soon as the input context needs to be updated. The maximum penalty is in the 

10 case where a new context value is inserted after each input data pattern. In summary, the introduction of the context 
feature in a multi-processors unit gives the flexibility to dynamically configure it either as a single unit or as separate 
sub-units thereof that can work independently without impacting the operation of others. The context approach as 
described above i.e. based on a totally hardware solution, is therefore a significant feature of the present invention. 

15 ' THE R/W MEMORY CIRCUIT (250) * 

Now turning to Fig. 18 which is comprised of Figs. 18(A). (B) and (C). Fig. 18(A) shows the detailed construction of 
RA/V memory circuit 250 comprising a 64x8 bit RAM memory 251 (usually referred to as the weight memory), address 
counter 252 and optionally a register labelled 253. Signal A represents the input vector components that are outputted 

20 from A register 13 on the DATA-BUS. Signal B represents the prototype vector components that are outputted from 
register 253 (if any) on the RAM-BUS. R/W memory circuit 250 may include the register 253 to latch the data that are 
outputted from the RAM memory 251 . As a result, the RAM memory 251 is isolated in the course of the automatic tests 
that are performed during the manufacturing of the ZISC chip 10. In this case, should the register 253 be used, then an 
additional register (not shown) placed in the input vector data path would be recommended to balance the two paths to 

25 the evaluator circuit 200 for pipeline operation purposes in the synchronous (BURST) mode. Register 253 is a free 
running registers, i.e. the data are loaded therein at each cycle, unlike the A register 13 whose loading is gated by a 
control signal. The role of address counter is to count the components of the input vector and to properly address the 
RAM memory 251. Counter 252 is controlled by two control signals: NEXTC (NEXTC stands for NEXT Component) to 
load the components in sequence) and FCOMP (FCOMP stands for First COMPonent) to reset the counter before 

30 loading the first component. Counter 252 is thus reset by the FCOMP signal at the beginning of a new recognition phase 
and incremented by signal NEXTC at each component presentation. Address counter 252 has been shown in R/W 
memory circuit 250 but could be placed anywhere in the neuron circuit architecture of Fig. 5. In reality, according to the 
present implementation of the ZISC chip 10. it is located in the status and control logic circuit 18. In this case, counter 
252 generates the RAM address signals that are then common to all the neuron circuits of ZISC chip 10. RAM memory 

35 251 is a one port RAM. The register 253 could be provided with a complemented output if so required by the processing 
of the B signal in distance evaluator circuit 200. The circuit which generates the RS signal will be described hereafter 
by reference to the daisy chain circuit 600. At this stage of the description, it just suffices to say that the RS signal which 
is applied to RAM memory 251 of neuron circuit 1 1 is inactive for all engaged neuron circuits and is only active for the 
first free neuron, i.e. the ready to learn neuron circuit, to allow the pre-charge of the input vector components in RAM 

40 memory 251 . The RS signal broadly corresponds to the conventional R/W signal that is of current usage in any standard 
RAM memory. When signal RS is active (RS = 1), the RAM memory 251 is written, i.e. the components are loaded in 
sequence under address counter control. On the contrary, when signal RS is inactive (RS = 0), the contents of the RAM 
memory 251 can only be read. Now turning to Fig. 18(B), there is shown the data flow for an engaged neuron circuit 
during the recognition phase. The components of the input vector A are only loaded in register 1 3 but not in RAM memory 

45 251 , because signal RS is inactive. None component of the input vector is therefore written in RAM memory 251 (the 
path to the RAM memory 251 is shown "blanked") of an engaged neuron circuit, because such writing would be detri- 
mental to the integrity of the prototype vector components stored therein. Signals A and B are available for comparison 
in distance evaluator circuit 200. Fig. 18(C) shows the data flow for the first free ready to learn neuron circuit still during 
the recognition phase. In this case, the components of the input vector are stored in RAM memory 251 as the weights 

50 of the new prototype vector when control signal RS is set active. As illustrated in Fig. 18(C) by the blanked paths, the A 
and B signals are not exploited by the distance evaluator circuit 200 in case of the ready to learn neuron circuit. The 
implementation of the pre-charge step thus requires the R/W memory circuit 250 and a circuit that is capable to generate 
the RS signal and designate the determined free neuron circuit whose weight memory must be loaded with the input 
vector components. With the present implementation of the base ZISC chip 10, the role of this circuit is fulfilled by the 

55 daisy chain circuit 600. In that regard, the R/W memory circuit 250 and the daisy chain circuit 600 thus form the "pre- 
charge circuit". This step of precharging the input vector components in the RAM memory 251 of the ready to learn 
neuron circuit during the recognition phase is an important feature of the present invention. Compared to the known 
solutions of the prior art, this pre-charge step has a major advantage because it avoids to present again the input vector 
during the learning phase, it saves a lot of time, should the ready to learn neuron circuit is confirmed for being engaged. 
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' THE DISTANCE EVALUATOR CIRCUIT (200) * 

Now turning to Fig. 19. there is illustrated the internal structure of the multi-norm distance evaluator block 200 ^^^^^ 
is basically comprised of two sub-circuits referenced 201 and 202 and optionally of a register 203. The 8-bit DATA-BUS 
whS wTs the components of input vector A) and the 8-bit RAM-BUS (which transpoils t^e^m^nents of pro- 
totype vector B) are applied to sub-blocks 201 while the local norm signal No is applied to both sub-blocks 201 and 202^ 
B^'ck 200 generates5Se distance signal D on the 14-bit DIST-BUS. Note that, -'.<=^°-f ";^:„22 P^sents 
input vector A whose all components are equal to zero, the distance evaluator 200 is able to °"*P"* 
he components of the prototype vector B on the DIST-BUS. This property will be used .n the so-called SAVE mode The 
RAM^o^iV rr^Lb-btocks 201 and 202 of distance evaluator circuit 200 form all together a three-stage pipelined 

'*'"ts mentioned above, according to a significant feature of the present invention, block 200 works as a processing 
unit a^TsXtSto^computethec^stance with either theL1(MANHATT^^^ 

that hS been optimized for using only a minimal number of devices. The local norm signal No which ,s transpor ed on 
,5 t^^-bfNo bSs "uSd to select the desired norm to be used in the distance computation. If the norm signal No is 
equal to 1. the distance between input vector A and prototype vector B is calculated according to the L norm and 
inversely if the norm signal No is equal to 0, said distance is calculated with the Lsup norm. Therefore, still us^g the 

notetS usi above i e A = A1, ... Ai An and B = B1. ... Bi Bn (each Ai or Bi component being coded on m 

bSHe ca^uS^^^ D will be either: sum(abs(Ai-Bi)) tor the LI norm or max(abs(Ai-Bi)) for the Lsup norm wi* 

0 - i n) Remark, abbreviation abs means "absolute value". The example below illustrates this statement. Let us 
assume four component vectors A and B represented by A = 7 1 8 2 and B = 6 3 4 5. In this case, the distance is equal 

to 1 0 with the LI norm and to 4 with the Lsup norm. . ^ ui i, oor. ^« niQT ri i<? i<; 

Let us assume for sake of generalization that the distance signal D generated 200 on the D STBUS 

comprised of bits dO ... . dp-1 . The number p of bits for the distance signal D depends on the selected norm. For the 
LSUP nom^ p = m (same number of bits), while for the LI norm, the bit number depends on the number of componente 
n For example, to n = 4. the maximum number given by the LI norm is (2'*8)*4 = 1 024 and therefore the result needs 
to be ided on 10 bits because 2"10 = 1024. In the described implementation of the ^^^'^^'^l'^-'^^'^^^^ 
components coded on m = 8 bits, the real number is (2"8)*64 = 2**14. which explains why the DISTBUS has a 14-bit 

"""^ Sub^lfoclf201 computes the absolute value of the difference between two corresponding compon^ts Ai and Bi of 
the input and prototype vectors respectively, i.e. (abs(Ai-Bi)). To compute the difference (A,-Bi) results in the computation 
of^e sum: (AU (Bi^) in an adder, and thus t^ 

B outputted by the (Sw memory circuit 250, a battery of m = 8 inverters 204 may be required in one input path of adder 
205. Adder 205 hasbeen specifically designedtomeettheneeds of the presentinvention in te^^^^^ 
Adder 205 computes the internal carry generated by bits 0 to m-1 . The signal which is outputted by ^dd^r 205 is appl.^ 
to the input of a two-way exclusive OR (or XOR) circuit referenced 206. The carry signal which is generat^ by adde 
205 it labelled Cgen ThL signal is applied to adder 205 as the input carry signal Cin and to another two-way XOR circuit 
2^7. whte othe? input is the local norm signal No. Depending on the value of local norm signal 
the polarity of the output signal X generated by XOR circuit 206. The innovative design o this specrfic adder 205 thus 
allocs to re-inject the "carry-generate" signal at the input carry terminal as the Cin signal. The base function of adder 

205 will be better understood with the following general explanation. . . .u » i„ «rHa,- 1« 
If Ai>Bi, the adder 205 produces an internal carry signal, i.e. Cgeni = 1, which 'f ^^^^^^ *° ^^^^nd ^^^^^^^ 

produce the correct value of (abs(Ai-Bi)). In this case, if No = 1, the result X, is equal to abs(Ai-Bi) and if No - 0, the 
result Xi will be the complement of this value i.e. aBslAT-Biy. This value Xi may be latched in register 203_ 
45 If Ai < Bi or Ai = Bi the adder 205 doesn't produce any internal carry signal i.e. Cgeni = 0. If No = 1 . he XOR c^ctnt 

206 complements the result in order to produce the correct value. If No = 0. this ^"'^P'^'^^^f °" .^j" 

The numeric examples given just below, based on 4-bit components for the two cases: Ai < Bi and Ai > B. (the case 
Ai = Bi being handled the same way as the case Ai < Bi). illustrate the point. 
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First case: Ai > Bi (e.g. Ai = 2 and Bi = 1) 



Ai=0010 ===> Ai=0010 
Bi^OOOl ===> §1=1110 



10 adder ===> 0000 and Cgeni = l ===> Cini^l 

Cini ===> 1 



If No=l: Xi=0001 (No XOR Cgeni)=0 i.e. Xi= abs(Ai-Bi) 

If No=0: Xi=1110 (No XOR Cgeni)=l i.e. Xi= abs(Ai=Bi) 



20 As apparent fronn the above computation, adder 205 generates the term (Ai-Bi) = 0000 and a carry Ggeni = 1, so that 
Cini = 1 . In turn, this value Cini is applied as the input carry to adder 205 and thus added to the result 0000 + 1 = 0001 . 
On the other hand, the Cgen carry is applied to XOR circuit 207 which performs the exclusive OR function betwegi the 
local norm signal No and Cgeni. Then, the result is applied to XOR circuit 206 with the partial sum Si generated by adder 
205. Finally, the signal labelled Xi which is obtained at the output of XOR circuit 206 is such as Xi = ((No XOR Cgeni) 

25 XOR Si). Finally. Xi thus corresponds to either abs(Ai-Bi) or abs^AT-Bi). 

Second case: Ai<BI (e.g. Ai = 1 and BI = 2) 



Ai=0001 ===> Ai=0001 
Bi=0010 Bi=1101 



adder -==> 1110 Cgeni=0 Cini=0 
Cini ===> 0 



If No=0: Xi=1110 (No XOR Cgeni)=0 i.e. Xi= abs(Ai-Bi) 

If No=l: Xi==0001 (No XOR Cgeni)=l i.e. Xi= abs(Ai-Bi) 



50 To summarize the function of sub-block 201 , it can be said that the signal Xi which is outputted therefrom, corresponds 
to either abs(Ai-Bi) if No = 1 (i.e. norm L1) and to its complementary value a5sXAi-Bi} if No = 0 (i.e. norm Lsup) for each 
component i of the input and prototype vectors. 

The role of second sub-block referenced 202 is to use signal Xi generated from sub-block 201 to compute the 
distance D. A free-running register 203 is preferably added between sub-circuits 201 and 202 should a pipelined process 

55 be used. 

The X signal is applied to one input of adder 208 whose other input receives the D signal. Adder 208 may have the 
same structure as adder 205 or be of any conventional type. Its role is two-fold: for No = 1 (LI norm), it sums the current 
value of X with the distance D, for No = 0 (Lsup norm), it compares the current value of X with the said distance. The 
carry out signal Cout generated by adder 208 is applied to an incrementer 209 driven by the local norm signal No. 
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incrementer circuit 209 operates normally if No = 1 (norrr, L1 ) and is totally .r^h.t.ted (,t 9f ^/f^^^J^^^^^^^ 
l^up) Note that, this incrementer 209 is only used for the LI norm. Thanks to th.s circuit the brt ange o^^^^der 208 
Sn be rSuced irom p bits to m bits irrespective the norm used. For each sum, the abs(A.-B.) value -s coded on m b.ts. 
so that only a m-bit capacity is needed for adder 208. , . * .on 

The X sianal is also applied to the first input terminal (1) of a selector circuit 210 via a battery of eight inverters 21 1 
(one^^e^tS pe bitrTheTutput of adder 208 is connected to the second input terminal (2) of selector 210 and f.nally. 
^si^^Jis Ip^ ed to the^third input terminal (3) thereof. The norm signal N and the Cout sigrnal are respecmjely 
appSTo the command (Cmd) terminals (1) and (2). In case the LI norm is used, ^^^f f'f^^' ^.^^^^^^^^^^ 
(2) When the Lsup norm is used, signal Cout makes the selection between input terminal (1 ) and (3). '"P"! f J"^^ ( j 
10 s us^ if X > D and input terminal (3) is used if X <= D. As a result, the selector 210 is used to choose the adequate 
esulfdfperiing u^^^^^ of the norm signal N and Cout signals. A register referenced 212 storesjhe current 

Sle duTng the ^^^^ -nd thus works as an accumulator. The D signal which is the result of the distance 

calculation is available at the output of register 212 on the 1 4-bit DISTBUS 



At the beqinning of the distance calculation process, the register 21 2 is set to zero. , ^ ^. . „ 

J VTnorrS is used (No =1). the adder 208 computes the sum between Xi = (abs(Ai-Bi)) and the ^'stance D ^^^^^^ 
stage of the process, or current distance, that is stored in the distance register. The f ^ ^ ° ^^^^^^^^^^^^^ 
+ D) available at input (2) to register 212 where it will be latched during the next cycle. If the sum. i.e '^e distance D 



20 



25 



inputi^t-io first and second input terminals of a register 212. When all the couples of components have been nputted 
Ihe reSter 212 contains the calculated distance D between input vector A and prototype vector B.according to the Li 



""'"i'f Lsuo norm is used (No = 0) the adder 208 computes the difference between abs(Ai-Bi) and the current distance 
D If m^dLrenc; cT^^^ a carry, i.e. Couti = 1 then. D > abs(Ai-Bi). In this case, input terminal (3) is se ected 

to el^ize t e m^^^^^^ the current distance and (Ai-Bi). If Couti = 0, then abs(Ai-BO > D [nput t|r-^^^^^^^^^^ 

seleS thisdifferenceis stored in register212.lnordertocompareabs(Ai-Bi)andD.a 

^ol^t^lTTe if it is require? to load abs(Ai-Bi) in register 21 2. this value must have the aPP-P"^e Pj^^^^^^^ 
this is the role of inverters 211. The present case will be more readily understood by reference to the two following 

examples. 
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Example 1 

abs{Ai-Bi)=0011 ==> Xi=1100 
D^OOOl ==> D=0001 



adder ==> HOI Cout=0 abs(Ai-Bi)>D 



40 



Example 2 



45 



abs(Ai-Bi)-0001 ==> xl=1110 
D=0011 ==> D=0011 



50 



adder ==> 0001 Cout=l ===> abs (Ai-Bi )<D 



in example 1 selector 210 selects Input terminal (1). the value Xi = aSslAT-BiJ is loaded in register 212. In 2, 
55 no aaSTs taken Signal Cout is thus the criteria that is used to identify wether Xi is strictly inferior to D or "Ot Nrte by 
the wTtl^^tttT^^^^^^^^ to suppress the inverters 21 1 . In this case, the distance register 21 2 comains .'"a J^s(;^B,y^ 
TheTefore in the next comparison step, at the adder input the polarity of the X signal must be modified in order to aHow 
me Smparison with the oStput of the adder. In this case the signification of the carry is opposite. The suppression of 
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the inverters make the process a little bit more complicated, but if m is a large number, this suppression saves a number 
of inverters. 

Now, an innovative fast and dense adder with carry generate fully adapted to meet the needs of adder 205 of Fig, 
19, such as implemented in theZISC chip 10. will be described. According to the novel implementation, adder 205 uses 

5 selector circuits in order to compute the internal carry (carry generate). The architecture that will be now described by 
reference to Figs. 20 to 23 is very efficient in terms of circuit density and speed (delay). 

As illustrated by Fig. 20, adder 205 can be split into 2 sub-circuits 213 and 214. Sub-circuit 213 uses primary input 
bits aO to a7 and bO to b7 for each component of the input and prototype vectors respectively to process them in order 
to compute intermediate signals generlcally designated by letters c (carry), f (force) and p (propagate) for convenience, 

10 and the internal carry Cgen. This signal Cgen is in fact the carry resulting of the addition without input carry Sub-circuit 
214 uses said intermediate signals outputted from sub-circuit 213 and an input carry signal Cin to compute thefinal sum 
signal S. Due to the innovative architecture of sub-circuit 213, it is possible to directly connect the output terminal Cgen 
to the input terminal Cin of sub-circuit 214 without any detrimental loop effect. 

Fig. 21 shows the detailed structure of sub-circuit 213. For each couple of bits forming a slice, e.g. aO and bO, a 

15 specific circuit e.g. 215.1 computes the "state" of this couple of bits i.e. it determines whether there is a carry modification 
or not, and if there is a carry modification, whether the carry c generated by this bit slice, is set to 1 or to 0. Fig.22 (A) 
shows the internal structure of circuit 215.1 which essentially consists of a two-way XOR referenced 216. This circuit 
21 5.1 generates two bits, labelled pO and cO, whose value is determined by the following rule. If pO = 1 . this means there 
is no carry modification, while if pO = 0. this means there is a carry modification. The cO bit is such as if there is a carry 

20 modification, then cO is equal to the carry Depending on design constraints, circuit 215-1 may be hard-wired differently 
so that different options are allowed. More generally ci may be made equal to either ai or bi thanks to adequate strapping 
connection as schematically shown in Fig. 22(A), or even to (ai AND bi) or (Ai OR bi) through adequate logic dotting 
function if electrical constraints justify to do so. 

Now turning again to Fig. 21 , circuit 215.1 processes bits aO and bO to generate pO=(aO XOR bO) and cO=aO which 

25 are outputted from sub-circuit 215.1 but they are also used internally for further processing. In the second level of sub- 
circuit 213. another specific circuit then processes two pairs of signals generated by respective circuits of the first level 
to compute the result "state" for a 2 bit slice. For instance, circuit 21 7.1 processes bits pO. cO generated by circuit 21 5.1 
and bits pi . c1 generated by circuit 21 5.2. Fig. 22(B) shows the internal structure of circuit 21 7.2 which basically consists 
of a selector 218 and a two-way AND gate 219 that are properly interconnected as shown. Circuit 217.1 generates bits 

30 c1-0 and f1-0 to be used both internally in sub-circuit 213 and in sub-circuit 214. Letters p and f are used as mentioned 
above to illustrate the polarity of the carry modification signal, having in mind that p = f. Therefore, because the circuit 
21 7.1 is depending upon the polarity of the input signals defined by bit p or f, preferably the circuit structure depicted in 
Fig. 22(C) is used instead of the circuit of Fig. 22 (B), when driven by input bit of the p type instead of the f type, for 
increased optimization (to save inverting gates). For instance, the circuit of Fig. 22(C) referenced 220.1 is used in the 

35 third level of sub-circuit 213 of Fig. 21 to generate bits c3-0 and p3-0. It is comprised of a selector 221 and a two-way 
NOR gate 222 with interconnections such as depicted in Fig. 22 (C). The operation of the two types of selectors may 
be basically understood from Fig. 22(D). Similar reasoning applies to the other half of circuit 213 which processes the 
four remaining bit pairs, so that circuit 220.2 (of the same type as circuit 220.1) generates bits c7-4 and p7-4. Finally 
the bit processing is continued through a last fourth level illustrated by circuit 223 which generates bits f7-0 and c7-0. 

40 These two bits are applied to a two-way AND gate 224 whose output bit represents the Cgen signal. Cgen is thus the 
internal carry for bits 0 to 7 of the primary input bits. This carry signal Cgen represents the internal carry of the primary 
bits assuming Cin = 0. Because the processing of said primary bits is performed in parallel, the delay is very small. 
Circuit 223 has an identical structure when compared to circuit 217.1. As apparent from Fig. 21 , in addition to bit Cgen, 
sub-circuit 213 also supplies all the intermediate bits mentioned above for subsequent processing in sub-circuit 214. 

45 The table given below summarizes the logic table of circuits 21 7.1 and 220.1. 
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Circuit 217.1 



10 



pl=l ===> cl-0=cO, 

if generated, the carry comes from bit 0 
cl-0=cl, carry from bit 1 



I.e. 

pl=0 



15 



20 



If pl=l and pO=l 



else 



_--> fl-0=0 (propagate carry 

through bits 1 & 0) 
^^^> fl-0=l (carry generated 

by bit 1 or 0) 



Circuit 220.1 



25 



30 



35 



£3-2=0 c3-0 = cl-0 

if generated the carry comes from bit 1-0) 
£3-2=1 ===> c3-0 = c3-2 carry from bit 3-2 

If f3-2 = l or fl-0=l =-> P3-0=0 (carry generated 

by bits 1 or 0) 



else 



p3-0=l (propagate carry 

through bits 1 & 0) 
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The computation ot all the sum bits sO to s7 of the sum signal S from said intermediate bits and the input carry bit Cin 

will be now f^,^ ^ur identical specific circuits referenced 225.1 to 225.4^ Each 

Now turning to Fig. 23. sub-circuit '^ Tirsiinuuueo ooTanri of one selector 228. For instance, referring 

and 227 are the sum bits si and sO respectively. Same ^Jf"" "^/^^^ to said first input terminal 

c6, and bits p2 to p7 are concerned, except however ,n either bit c1 0 or bit Cin 

of XOR 227 of circuits 225.2 to 225.4. The bit «^'^^'^^«P£^%^°^ A sim a instruction applies to selector 229.2 
depending the value of control bit f 1 -0 which is applied ^ selector ^9^^ TfomDlS^ with respect to circuit 229.3. 
where bit p3-0 selects either bit c3-0 or bit Cin. The situation is a little bit more ^°"^P'^^*^^J";.^J„,e,ed in two-way 
The introl bit applied to selector 229-3 is equal to -^''f ^^^^^^ c5-I 
NAND gate 230. This control bit f5-0 selects either bit C.n or f ^ f Z^le6 232 generates the carry 

or C3-0 in selector 231 under the control of bit f5-4. AfappL J^^^^^ construction shown in 

out bit Cout, which is either bit Cin or bit 07-0 under to the pair of sum 

Fig 23. circuits 225. 1 to 225.3 respectively delivers pairs of sum bits s2 s3, s4 s5, sb s/ in aao 
bits so si generated by circuit 225.1 as explained above. 
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With the implementation of sub-circuits 21 3 and 214 respectively illustrated in Figs. 21 and 23. the critical delay can 
be split In two parts. Sub-circuit 21 3 needs 4 stages to generate said intermediate signals, but 5 stages are necessary 
to generate signal Cgen. Sub-circuit 21 4 needs 4 stages. However, depending on the application, it is possible to reduce 
the number of elementary circuits in sub-circuits 213 and 214. but at the cost of increasing the number of stages and 
5 thus the overall delay. 

* THE DISTANCE COMPARE AND IF CIRCUITS (300/350) * 

Compare circuit 300 is implemented in each neuron circuit 1 1 to perform the comparison between the calculated 
10 distance signal D and the J signal. The J signal is supplied by IF circuit 350 shown in Fig. 7. As apparent from Fig. 7. 
the J signal is the signal which appears either on the DIST-BUS. the OR-BUS, or the MinlF/MaxIF-BUS or the content 
of the AIF register 352 which holds the AIF value. Fig. 24 shows the detailed structure of the fully parallel compare circuit 
300 according to a preferred implementation of the ZISC chip 10. Now turning to Fig. 24. compare circuit 300 receives 
the 14 primary bits dO to dl3 of the distance D signal generated by the distance evaluator circuit 200 on a first set of 
15 input terminals. It further receives the 14 primary bits jO to j13 of the J signal from the IF circuit 350 that are applied on 
another set of input terminals. In essence, the compare circuit 300 generates two signals LT (Lower Than) and LTE 
(Lower Than or Equal) which in turn are applied to the identification circuit 400. The general principle of the comparison 
between the D and J signals performed in compare circuit 300 is to determine (1) if D is strictly inferior to J (this is the 
role of the LT signal) and (2) if D is inferior or equal to J (this is the role of the LTE signal). 
20 During the recognition phase, signal J represents the value stored in the AIF register 352. The calculated distance 
D is compared to the AIF value to determine whether D < AIF or not. On the contrary, during the teaming phase, signal 
J is equal to MinlF The calculated distance D is then compared to MinlF to determine whether D <= MinlF or not. The 
following TABLE I illustrates the conclusions with regard to adetermined neuron circuit 1 1 that may be reached depending 
upon the values of the LT and LTE signals in their respective mode. 

25 

TABLE I 



In the RECOGNITION mode: 


LT 


conclusions 


. 0 


no firing (D >= AIF) 


1 


firing (D < AIF) 


In the LEARNING mode: 


LTE 


conclusions 


0 


not degenerated (D > MinlF) 


1 


potentially degenerated (D <= MinlF) 



40 

To that end, a comparison is performed between each corresponding bits of signals J and D. Thanks to the arbo- 
rescent architecture depicted in Fig. 24 which allows a parallel processing of the signals, the comparison is performed 
very fast. For each couple of the primary bits forming a slice, e.g. dO and jO, a specific circuit e.g. 301 .1 computes the 
"state" of this couple of bits i.e. it determines whether the two bits are equal or not. Circuit 301 .1 is structurally similar 

45 to circuit 215.1 of Fig.22 (A) which essentially consists of a two-way XOR referenced 216. but it further requires an 
inverting gate at its output to perform the desired XORNOT (XORN) function. This circuit 301.1 generates two bits, 
labelled pO and cO. whose values are determined by the following rule: if pO = 1 . this means there is an equality between 
bits dO and jO and if pO = 0. this means the two said bits are different. As apparent from Fig. 24, the cO bit is made equal 
to do. Depending on design constraints, circuit 301 . 1 may be hard-wired differently Circuit 301 . 1 processes first pair of 

50 bits do and jO to generate bits pO and cO. Similar circuits 301 .2 to 301 . 1 4 process the other pairs of bits i.e. d1 and j1 to 
d13 and j13 in order to generate respective pairs of bits pi and c1 to p13 to c13. 

In the second level of circuit 300. another specific circuit then processes the two pairs of bits generated by respective 
slice of circuits of the first level to compute the result for a two bit slice. For instance, circuit 302.1 processes bits pO, cO 
generated by circuit 301.1 and bits p1, c1 generated by circuit 301 .2. In turn, circuit 302.1 generates pair of bits c1-0 

55 and f 1-0- Circuit 302.1 has an identical structure to circuit 2 17.1 shown in Fig. 22(B) whichbasically consists of a selector 
218 and a two-way AND gate 219 that are properly interconnected as illustrated therein. Same construction applies to 
circuits 302.2 to 302.7. Because circuit 303.1 is depending upon the polarity of the input bits defined by letter f instead 
of p, preferably the circuit structure depicted in Fig. 22(C) is used instead of the circuit of Fig. 22(B), for increased 
optimization, i.e. to save inverting gates. Therefore, circuit 303.1 is preferably identical to circuit 220.1 of Fig. 22(C) to 
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generate pairs oi bits f7-0, c7-0 and ^^^■^■'^' -^ '^^^r^T^^^^^^ bits p13-0 and c13-0. Circuit 305 has an 

22(B). Finally, these pairs of bits are applied to ^^"^f ^ are applied to a circuit 306 which is first 
identical structure when compared to c^cu^ St feSesInJth^^ signal. Bite pl3-0 and LT are applied to a 
comprised of a two-way NOR ^at^^OT whose ou^^^^^^^^ J „«, m 

two-way OR gate 308 to generate ^^^^^'^^l^^^l^^^^ 

SSilTe^P^^^^^^^ 
of devices. 

* THE IDENTIFICATION CIRCUIT (400) * 

MOW .rnlng to Fig. 25. there -hown th^d^i^^^^^^^^^^ 
14-bit CAT-BUS (Which transports the .nput ^^^^ ^,fJ^;;9^to^^ whose output is connected to 

(Which transports the global category C* signal) are apphed to a^o way mu P ^^^^ ^^^^^^^^^ ^^^^^ .^^^ 

Ihe first input of comparator 402. Depending on he o er^^^^^^ 

category signal CAT or the global category signal C network 11(A). In this case, the 

can be replaced by a main multiplexer ^'JJ"" AT BU^co^^^^^ CAT or C* data. The Cat-BUS is applied to 

OR-BUS would be no longer necessary. is3eS^^^^^^^ made of fourteen 2-way XOR gates driving a 

the second input of comparator 402. Compara^^^^^ 

single 1 4-way OR gate. In turn, the output o comparator ^O^^^appuea ^TL-BUS and in fact, directly denves 

other input receives the LEARNING signal L. S'S^^' J:^^^^^^^ 22 control. The L signal is 

from a signal generated in the status and cor^ro. log c arcurt 1 8 unde^m c ^^^^ ^^^^^^ 

set active during the learning phase (L = 1) and ^et -nacftve (^^ learning phase pertains to identification circuit 400, 
decision that a neuron circuit must st°^«^^";"P"^:jt .^^^^^^ of the ready to learn neuron circuit, as it 

the role of the daisy chain circuit 600 ^^^^ ' e SJai^^^^^^^^^^^^ XOR gate 403 is labelled the K signal, 

will be explained hereinbelow in more ^ils. The one brt signal^g^^^^^^ ^^.^^ ^^^.^ 500 

The LT and CO signals are applied to a two-way 9^*^ outputted from AND gate 404 is preferably 

is set active (CO = 1) as soon as the neuron circuit l'^^^^^;"";,^^^^^ signal which is generated by the 

stored in an optional one-bit F '""^^^^ hasfiredintherecognijon 

register 405 is the local fire result signal F p'S"^^'^;^^^^* J^'^^^^^^ are applied to a three-way AND gate 406. The 
mode. Signals LTE and CO and the ^'f 1°"*^^^^^^^^^^^^ tS control of signal L and is outputted from 

signal generated by AND gate 406 is stored '"J^b J ^EG J« " ,,,,.,t , , , Register 407 holds a "1 " 

identmcation circuit 400 as the DEG ^'Q^^S 9njm^°^^ AIF?egiJer 352 is loaded with MinlF). The signal which is 
when the neuron circuit 11 is Regenerated (n^ h s cas^^^^^ p,^„,e ,he 

outputted from register 407 and *J ^ Jtn^^^^^^^^^ degenerated neuron circuit has fired during 

.0 DEGOUT local result signal. -T^^^.^^^O^^^^""' "J'^^^^^^ to a tvTo way AND gate 409 to generate the local result 

the recognition phase. Finally, signals F and K a-^^/Pf^^^ *° ^^^^^^ ^ operating mode. In the learning mode, during 

UNC/FIRE.OK signal. This signal has two meanings d^P^"^ °" is set active (UNC/FIRE.OK = 

the engagement process of the ready *o J-r--^^^^^ Vhe re^d^toSfn 'e-on circuit is not engaged because 
1)thestoredinputvectorisnot "frozen asapro^^^^^^ 

45 the input vector has been recognized by at least one "e"'°" ' cateaories have fired, i.e. the input vector has 
UNC/FIRE.OK signal is set active « ^ r ^nTcS ^T ceSl t^^^^^^^^^^ signals F DEGOUT and 

Sc^FrREr^^^^^^ 

a'dTnLTt^sSiDEGI^^ 

AS a matter of fact, in the recc^n.tK.n -^J^f^^";^^^^^^^ global category sign^ C* presented on 

C stored in the category register 450 of a ^^^^-^'"^ ^j,, be no matching in comparator 402. Let us 

the OR-BUS. If there is a difference between f f ^^^^"^J^^^^"^'^^^^^^^^^ this mode, the signal L is inactive (L = 0), the 
assume a logic one is generated in this case for "J^ .^f (F = 1), sTgnal UNC/FIRE.OK is equal 

Signal K outputted from XOR gate 403 will be thus set to 1 and J^"^ es'^aSL. At ieaS one UNC/FIRE.OK signal 
to 1 . This means that at least two ^'[^"'^/I^J^^^^^ of the ORing of all local result signals is 

being set to 1 , as a consequence the global result UNC/FIRE^OK ^^^^^^^^^ recognized by at least two 

sett? 1. At the base neural network ^ ^ t'^) 'ff^^^'^^^f '^^^^ 

committed neuron Circuits that have fired with difteren c^^^^^^^^^^ P ^ identical to the global 

On the contrary, if all committed neuron circuits that have fired have eacn a -a 



30 



35 



50 



55 to 1 



32 



BNSDOCI0-<EP 0694855A1> 



EP 0 694 855 A1 



category C*, all the UNC/FIRE.OK local result signals that are generated therefrom are set to 0. and therefore the global 
result signal UNC/FIRE.OK* is equal to 0. In the latter case, the input vector is identified and its category is equal to C*. 

In the learning phase, it is important to determine whether the neuron circuit will become degenerated or not. For 
an engaged neuron circuit, the input category CAT supplied by the micro-controller/user 22 on the CAT-BUS is compared 

5 to the local category C in comparator 402. If C is equal to CAT. UNC/FIRE.OK = 1 and signal LTE is not used, the 
reduction process is not undertaken because this neuron circuit has recognized the input vector with the correct category. 
If C and CAT are different, two cases are to be considered depending on the result of the comparison between distance 
D and the value of MinlF. in other words of the signal LTE value. If signal LTE = 1 (i.e. D <= MinlF), neuron circuit 1 1 is 
declared degenerated, the logic one generated by the AND gate 406 is stored into the DEG register 407 and the MinlF 

10 value is stored in AIF register 352. If signal LTE = 0 (i.e. D > MinlF). the contents of the DEG register 407 is not modified, 
the signal outputted by AND gate 406 being equal to zero. In addition, at the neural network level, comparator 402 has 
the role of determining whether the input category CAT has been recognized by at least one neuron circuit. To determine 
whether neuron circuit 1 1 is degenerated or not is an essential step of the reduction process. In the learning phase 
(signal L = 1), if the K signal outputted from XOR gate 403 is set to 1 (because the input category CAT is the same as 

15 the local category C and there is thus a matching in comparator 402) and if signal F = 1, then signal UNC/FIRE.OK is 
set to 1 . The UNC/FIRE.OK signal generated by AND gate 409 has the meaning of indicating that the neuron circuit in 
consideration has correctly recognized the input vector, and therefore the input vector has been correctly recognized at 
the base neural network 11 (A) level. In this case, the ready to learn neuron circuit still remains free and the stored input 
vector is not "frozen" as a new prototype vector in RAM memory 251 . There is no need for the neural network to engage 

20 a new the ready to learn neuron circuit. 

It is to be noted, that neither during the recognition phase nor during the learning phase, the UNC/FIRE.OK signal 
which results of the comparison between the global signal C* and local signal C in identification circuit 400, is locally 
exploited unlike global result signal UNC/FIRE.OK*. In summary, in the recognition mode, signal UNC/FIRE.OK?^= 1. 
means that at least two neuron circuits with two different categories have recognized the input vector. Signal 

25 UNC/FIRE.OK* = 0 means the input vector has been either recognized by at least one neuron circuit with a single 
category or not recognized at all. In the learning phase, the global result signal UNC/FIRE.OK* (via the ST signal which 
derives therefrom) has the determining role to force the ready to learn neuron circuit to freeze or not the input vector 
that has stored in RAM memory 251 during the pre-charge step that has been completed during the previous recognition 
phase. 

30 

* THE Dmin DETERMINATION CIRCUIT (500) * 

As mentioned above by reference to Fig. 8, the Dmin determination circuit 500 is comprised of multiplexer 501, 
search/sort circuit 502 and logic circuit 503. Fig. 26 schematically shows the internal construction of logic circuit 503 

35 which generates the SiL"ECT signal which is applied to search/sort circuit 502. In reality, logic circuit 503 not only 
receives the F. CO and RS signals as apparent from Fig. 8, but it also receives three control signals that are related to 
different operating modes/sub-modes of the base ZISC chip 10: the NORMAL (NM). the NEAREST NEIGHBOR (NNM) 
and the SAVE (SM) signals. These three control signals are generated by the status and control logic circuit 18 and 
transported via the lADD/CTL-BUS to logic circuit 503. Logic circuit 503 comprises one 3-way AND gate 503.1, two 2- 

40 way AND gates 503.2 and 503.3, and one 3-way NOR gate 503.4. The CO. F and NM signals are applied to AND gate 
503.1 . The CO and NNM signals are applied to AND gate 503.2. The RS and SM signals are applied to AND gate 503.3. 
The outputs of the said three AND gates are applied to their respective input of NOR gate 503.4 which generates the 
SELECT signal. In turn, the SELECT signal is applied as an inhibit signal to search/sort circuit 502 as it will be illustrated 
hereinbelow by reference to Fig. 30. 

45 In essence, the primary role of search/sort circuit 502 is to participate to the searching of the minimum distance 
Dmin among all the distances calculated by neuron circuits of a neural network and then to participate to the sorting of 
the remaining distances in an increasing order (it is also adapted to handle category type of data). Therefore, it is impor- 
tant to notice that the processus of searching and sorting are collectively conducted by all the neuron circuits of the 
neural network. More generally, the role of search/sort circuit 502 is first to find the minimum value among a group of 

50 several parameters and then to sort the remaining values in the increasing order. However, its adaptation to find the 
maximum value and then to sort the remaining values in the decreasing order would be trivial for the man skilled in the 
art. The search and sort processes must be conducted in the shortest time and with a circuitry as simple as possible. 
These criteria are met with the search/sort circuit 502 of the present invention. Only the search process to determine 
the minimum distance Dmin and the related circuitry will be discussed hereinbelow. However, the search process for 

55 the maximum distance determination would basically follow the same rules and use a quite similar circuitry. 

The algorithm at the base of the search process of the present invention will be briefly described by reference to 
the flow-chart referenced 504 shown in Fig. 27, Assuming four neuron circuits 11-1 to 1 1 -4, each having calculated its 
respective distance D1 , D2. D3, and D4 between the input vector and its stored prototype vector. The problem which is 
raised is thus to determine the lowest value Dmin among these four distances. First of all, a matrix is built wherein the 
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to 4) is the rank of the neuron circuit in consideration. As apparent from TABLE 1 . to ^acMme also co e po 
word (e.g. L1 which is formed by the MSBs of each distance word is such as L1 = 0110). 

TABLE II 



10 



15 





D1 


D2 


D3 


D4 


LI 


d11 =0 


d12= 1 


d13 = 1 


d14 = 0(MSB) 


L2 


d21 = 1 


d22 = 0 


d23 = 1 


d24= 1 


L3 


d31 = 1 


d32 = 0 


d33 = 0 


d34 = 0 


L4 


d41 =0 


d42= 1 


d43 = 1 


d44 = 1 (LSB) 



20 



25 



column, as schematically illustrated by the two next lines: 



30 



Dl D2 D3 D4 



LI 



0 ==> 0 found 



35 



SL 



40 



2 d^i^ot Change the previous status line SL as illustrated below, because line L2 does not .nclude any zero. 



45 



Dl 



D2 



D3 



D4 



50 



L2 



-=> 0 not found 

! 



SL 



X 



1 
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The process continues for line L3. and during step 3. the presence of a new zero is detected in distance D4 for bit d34. 
so that distance D1 is now excluded, as illustrated below. 

Dl D2 D3 D4 

5 



L3 1 X X 0 ==> 0 found 

10 

\ 

SL X X X < ! 

15 

Finally, in step 4, line L4 is examined in box 506, no zero being found, box 507 is by-passed and thus the status found 
in step 3 is confirmed. Distances D1 . D2. D3 being excluded, the minimum value is therefore represented by distance 
D4 found by neuron circuit 11-4. 

20 

Dl D2 D3 D4 

25 L4 X X X 1 ==> 0 not found 



SL X X X < ! 

30 

Because line L4 is the last line, the searching process terminates in box labelled "END" after the test performed in box 
508 Finally, the minimum distance is given by: Dmin = D4 = 01 01 . 

35 Fig. 28(A) shows the schematic construction of the elementary base unit, generically referenced 510, which is the 
corner stone of the search/sort circuit 502 according to the present invention. As apparent from Fig. 28(A), reference 
51 0-ki designates the elementary base unit which processes bit dki (line k, column i) of the distance signal Di previously 
computed by evaluator circuit 200 of neuron circuit 1 1 -i (with k = 1 to p and i = 1 to N). Basically, unit 510-kl consists of 
two sub-units 51 1-ki and 512-ki. Three input signals are applied to unit 510-ki : input bit signal dki, exclude input signal 

40 EXCLINki. and feedback signal ORk and two output signals: local output signal NOUTk-i and exclude output signal 
EXCLOUTki are outputted therefrom. Depending its value, the EXCLINki signal is used to select or to inhibit unit 510- 
ki for the search process, i.e. the Dmin determination. For instance, if EXCLINki is set inactive (EXCLINki = 0), unit 510- 
ki is not excluded and conversely, if EXCLINki is set active (EXCLINki = 1). unit 51 1-ki is made inoperative, NOUTk-i is 
set inactive (NOUTk-l = 0 with the present implementation of the ZISC chip 10 based on OR circuit 12). Basically, the 

45 function of sub-unit 51 1-ki, when not inhibited, is to search whether bit dki is at "0" or not. To that end. a specific logic 
function is performed between the bit signal dki and the EXCLINki signal to generate the output signal NOUTk-i. On the 
other hand, signal EXCLOUTki is a determined logic function of signals EXCLINki, dki and signal ORk. The role of unit 
512-ki is thus to generate an adequate input signal for the following unit 510-(k+1)i pertaining to the next line which 
processes bit d(k+1)i to inhibit or not said unit depending on its value. In other words, sub-unit 512-ki determines if 

50 distance Di will be excluded or not when the following line is processed. Signal ORk which intervenes here as a gating 
signal, results from the ORing of all the NOUTk-i signals generated by line k (ORk = NOUTk-1 OR ... OR NOUTk-N). 

In a preferred embodiment illustrated in Fig. 28(B), subunit 51 1-ki consists of a NOR gate 513 which performs 
NOUTk-i = dki NOR EXCLINki. Sub-unit 512-ki which plays an essential role as to the exclude function is comprised of 
AND gate 51 4 and OR gate 51 5, that are interconnected to perform an AND-OR function, so that EXCLOUTki = EXCLINki 

55 OR (dki AND ORk). For instance, if signal EXCLINki = 1 , then signal NOUTk-i is forced to zero irrespective the value of 
dki to be neutral with regard to OR circuit 12. Now. if signal EXCLINki = 0 (no exclusion), signal NOUTk-i will be equal 
to cJkT. If dki = 0. then signal NOUTk-i = 1 , the latter value is therefore characteristic that a zero has been found at this 
location. As a consequence, signal ORk is equal to one, and because dki = 0 and signal EXCLINki = 0, we have signal 
EXCLOUTki = 0. This means, that the following unit will not be inhibited. On the contrary, if dki = 1, because signal 
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EXCLINki is still equal to 0. signal NOUTk-i is set to 0. Now, two cases must be considered depending on the results 
obtained by the other non excluded columns for this line k. If all the input bit signals are equal to one. then all the local 
result signals of this line k (for all the values of i) are equal to zero, then signal ORk = 0 and finally, signal EXCLOUTki 
= 0 (no exclusion). If, still for line k. at least one input bit is equal to 0, then the corresponding local result signal NOUTk- 
5 i is equal to 1 , so that signal ORk = 1 and finally. EXCLOUTki = 1 , the column i is thereby excluded. 

Let us now consider Fig. 29 which shows how four search/sort circuits 502-1 to 502-4 (belonging to four different 
neuron circuits 1 1 -1 to 1 1-4) are combined via the adequate dedicated OR sub-circuit of OR circuit 12 to generate bit 
OUT*1 to OUT*4 of global output signal OUT*. The following description will be made by reference to the example of 
TABLE II with the ZISC chip 10 implementation of Fig. 4(A). This OR sub-circuit of OR circuit 12. which handles the local 
10 output NOUT signals, bearsnumeral 12.1 inFig.29tofacilitatetheunderstandingofthesearch/sortcircuit502operation. 
For sake of simplicity, the search/sort circuits 502 illustrated in Fig. 29 have been voluntarily restricted to the circuitry 
that is necessary to perform the search function. The circuit shown in Fig. 29 which bears numeral 516 includes four 
search/sort circuits 502-1 to 502-4 that have been shown adjacent in Fig. 29 to form the aggregate circuit 517 and 
dedicated OR sub-circuit 12.1. Aggregate circuit 517 is thus comprised of sixteen elementary base units referenced 
510-1 1 to 510-44 that are arranged to form a matrix. It must be understood that each of these four search/sort circuits 
is located in a different neuron circuit and they have been only fictitiously assembled for sake of illustration. As still 
apparent from Fig. 29, any search/sort circuit 502 of neuron circuit 1 1 is built by piling or cascading the adequate number 
of these units 510 arranged in a column, in this instance four. Note that the logic level of each node of circuit 516 has 
been indicated in Fig. 29 to still comply with the example illustrated by TABLE II at the end of the Dmin determination 
(search) process All search/sort circuits are identical and have a same construction. Aggregate circuit 517 will be now 
described in more details. The first line of units referenced 510-1 1 to 510-14 is concerned by the respective MSB bits 
d1 1 to d14. In the following part of the description, signals EXCLINIi and EXCLOUT4i will be labelled EXCLIN-i and 
EXCLOUT-i respectively Similar reasoning applies to other columns. The exclude signals EXCLIN-1 to EXCLIN-4 are 
applied to units 51 0-1 1 to 51 0-1 4 respectively In turn, said units respectively generate the NOUT1 -1 to NOUT1 -4 signals 
that are applied to OR gate 12.1.1 which in turn supplies output signal OUT*1. Signal 0UT*1 is reinjected in each of 
units 510-1 1 to 510-14 of the first line as the OR1 signal. Same reasoning applies to the generation of the 0UT*2 to 
OUT*4 signals via respective OR gates 1 2. 1 .2 to 1 2.1 .4, which are also a part of dedicated OR sub-circuit 1 2. 1 . In reality, 
in the present implementation of the ZISC chip 10. dedicated OR sub-circuit 12.1 comprises OR gates 12.1 .1 to 12.1 .N. 
Units 510-1 1 to 510-14 also generate the intermediate EXCL0UT1 1 to EXCLOUT14 signals that are applied to the 
respective EXCLIN input terminals of units 510-21 to 510-24 of the second line. Same reasoning applies to the con- 
struction of the other lines of aggregate circuit 51 7 until the fourth line. In the fourth line, units 51 0-41 to 51 0-44 generate 
final EXCL0UT41 (or EXCLOUT-1) to EXCLOUT44 (or EXGLOUT-4) signals that will used later on. Note that, because 
of technology constraints. NOR togic gates may be chosen instead of the OR gates shown in Fig. 29. In this case, an 
inverter would be required for each NOUTk-i signal to retrieve the correct polarity of the ORk bit signal. In any case, the 
35 logic relation ORk = OUT*k = NOUTk-1 OR ... OR NOUTk-N must be fulfilled. 

Let us describe now the operation of the Fig. 29 circuit. All EXCLIN-1 to EXCLIN-4 signals are at the non inhibit or 
select logic level, for instance, they are set to zero still to comply with the above described example, i.e. EXCLIN-1 = ... 
= EXCLIN-4 = 0 Then, as soon as the search process is initiated, bits d1 1 to d44 are applied to their respective inputs 
of units 51 0-1 1 to 51 0-44 in accordance with their values given in TABLE II. As apparent from Fig. 29. all the MSB's, i.e. 
40 d11 d12 d13 and d14 forming binary word LI (LI =0110) are applied to their respective units 510-11 to 510-14 of the 
first'line According to the internal structure of the unite 510 as described above, the binary word which is outputted 
therefrom is N0UT1-1 NOUT1-2 NOUT1-3 NOUT1-4 = 1001 , i.e. Li. These NOUT signals areORed in OR gate 12.1.1 
to generate the first bit of signal OUT* labelled OUri . Signal 0UT*1 = 1 because at least one zero has been identified 
in the first line In tum. the OUT*1 signal is fed as a feedback signal into each of units 510-11 to 510-14 via a feedback 
45 loop This feedback signal is labelled OR1 for consistency with the above description. Next, the sub-unit 51 2 of each of 
the unite of the first line determines the value of the corresponding EXCLOUT signal. In this instance, the result is given 
by EXCLOUT1 1 = 0 EXCLOUT12 = 1 , EXCLOUT13 = 1 and EXCLOUT14 =0. These signals are applied to respective 
unite of the second line. i.e. 510-21 to 510-24. but only unite 510-22 and 510-23 are inhibited because their respective 
columns are excluded. As soon as an EXCLIN signal is equal to 1 , the corresponding unit 510 is inhibited and in torn 
50 Will inhibit the corresponding unit 510 of the following line pertaining to a same column by generating an EXCLOUT 
signal equal to 1 . Similar reasoning applies to other lines. The above described operation is continued line by line until 

the fourth and last line is reached. ... ^ ^ i 

TheEXCLIN41 signal applied to unit 51 0-41 is equal to 1 to indicate this column has been inhibited due to the value 
of signals 0UT*3 and d31 . There is no modification to the current status brought by the processing of the LSBs in the 
55 fourth line. The final result of the search process, which is outputted by OR gates 12.1.1 to 12.1.4. is given by signal 
OUT* = OUT*1 0UT*2 0UT*3 OUT*4 = 1010. Signal OUT* is equal to the complement of the distance D4 calculated 
by neuron circuit 1 1 -4. It may also be noted that all the EXCLOUT-1 to EXCLOUT-3 signals are at "1 " and only EXCLOUT- 
4 is at "0" In summary, not only the minimum distance Dmin = D4 has been determined (in fact, its complemented value 
Dmin" has been generated with the implementation shown in Fig. 29) but also the neuron circuit which holds it (in this 



15 



20 



25 



30 



36 



BNSDOCID <EP 0694855A1> 



EP 0 694 855 A1 



case, neuron circuit 11-4 is the only neuron cicuit whose EXCLOUT signal is equal to zero). The totality of the OUT* 
signal is available at the output of circuit 516 (which results from the serial connection of the aggregate circuit 51 7 formed 
by the search/sorter circuits of the four neuron circuits 11-1 to 1 1-4 and OR sub-circuit 12.1). The reasoning may be 
generalized to the assembly of N search/sort circuits 502-1 to 502-N. each handling p bits, to result in an aggregate 

5 circuit 51 7 now capable to determine the minimum distance Dmin among N distances D1 DN. Each distance being 

coded on p = 14 bits with the present implementation of the ZISC chip 10. In this case, OR sub-circuit 12.1 is comprised 
of p N-input OR gates 12.1.1 to 12-1.pto generate signal OUT* (comprised of bit signals OUT* 1 to OUT*p). OR signal 
is then comprised of p bit signals OR1 to ORp. that are directly derived from the OUT*1 to OUT*p bit signals. The above 
description relating to only the search process of the minimum distance Dmin has been given for sake of illustration, it 

10 must be understood that in reality the innovative search/sort circuit 502 of the present invention has been normally 
designed with both search and sort capabilities. 

As a matter of fact, the circuit of Fig. 29 can be significantly improved at the cost of only a small additional circuitry 
to add the sort function, so that the remaining distances are sorted in the increasing order If so desired. The search/sort 
circuit 502 with both capabilities is referenced 518 in Fig. 30. As apparent from Fig. 30, with respect to aggregate circuit 

15 51 7 and OR sub-circuit 12.1 (previously described by reference to Fig. 29), the additional circuitry includes two circuits 
referenced 519 and 520. For instance, as far as distance D1 and search/sort circuit 502-1 are concerned, dedicated 
circuit 519 includes a two-way OR gate 521-1, a two-way OR gate 522-1 and a storage element, such as a latch 523- 
1 . OR gate 522-1 and latch 523-1 are connected in series, the output of the OR gate driving the Input of the latch. The 
output of the latch 523-1 is connected in a feedback loop to one input of OR gate 522-1. The other input of OR gate 

20 522-1 receives an XOR-1 signal whose nature will be explained later on. The latch 523-1 receives the appropriate 
CLOCK and RESET signals as standard. The output of latch 523-1 is also connected to_the first input of OR gate 521- 
1 to apply the EXCLIN'-1 signal. The other input of OR gate 521-1 receives the 5ELEt;T-1 signal. The signal which is 
outputted from OR gate 521 -1 is applied as the EXCLIN-1 signal to unit 510-11 of search/sort circuit 502-1 . Still referring 
to the distance D1 column, circuit 520 includes an inverter referenced 524-1 connected to the output of search/sort 

25 circuit502-1 which generates the EXCLOUT-1 signal. The role of inverter 524-1 is to complement the EXCLOUT- 1 signal 
to produce a signal labelled XOR-1 . The XOR-1 signal is applied to the other input of OR gate 522-1 . As apparent from 
Fig. 30. circuits 519 and 520 include three other identical circuits for columns D2 to D4. In reality circuits 519 and 520 
are comprised of N of such identical circuits for the N columns corresponding to distances D1 to DN. 

Operation of the Fig. 30 circuit may be understood as follows, still complying with the TABLE II example. At initiali- 

30 zation. the EXCLIN'-1 to EXCLIN'-4 signals are reset to 0000. This is performed through the specific RESET control 
signal which is applied to each of latches 523-1 to 523-4, and let us assume that all the four sorters 502-1 to 502-4 have 
been not been inhibited by signal SELECT (SEL'ECT = 0) generated by logic circuit 503 (Fig. 26). As a result, signals 
EXCLIN-1 to EXCLlN-4 are equal to zero (no exclusion). The search process is conducted as described above until the 
minimum distance which corresponds to distance D4 determined by sorter 502-4 of neuron circuit 1 1-4 is found-with 

35 the net values indicated In Fig. 29. As apparent from Fig. 29. the EXCLOUT-1 to EXCLOUT-4 signals are represented 
by the binary word 1110. After inversion in inverters 523-1 to 523-4 of circuit 520, the corresponding XOR-1 to XOR-4 
signals are given by the binary word 0001 as shown in Fig. 30. The position of the "1 " indicates that the minimum distance 
is held by neuron circuit 1 1 -4. Let us now see the sort process in details. The XOR-1 to XOR-4 signals are ORed in OR 
gates 522-1 to 522-4 with the EXCLIN'-I to EXCLIN'-4 signals (equal to binary word 0000) and then latched in latches 

40 523-1 to 523-4 respectively Finally, the new value of the EXCLIN-1 to EXCLIN-4 signals is then given by binary word 
0001 which is applied to aggregate circuit 517. Because, the EXCLIN-1 to EXCLIN-3 signals are now equal to 0. the 
corresponding search/sort circuits 502-1 to 502-3 are permitted to participate to the next step of the sort process. Only 
search/sort circuit 502-4 does not participate to the following searching steps because distance D4 is excluded. Circuit 
518 re-initiates a search process to determine which of the three remaining distances D1. D2 or D3 is the minimum 

45 distance. The same processus as described above is conducted. Distance D1 Is known to be that searched minimum 
distance. Next, circuit 518 continues the search process to exclude the column of distance D1. The search and sort 
processes are continued as described above by reference to distance D1 . The distances are then made available sequen- 
tially in the increasing order, i.e. in the present instance, D4. and then D1 , D2 and D3. on the OUT*-BUS. Same reasoning 
would apply to a circuit adapted to search the maximum distance and then sort the remaining distances in the decreasing 

50 order Basically, with respect to circuit 518 of Fig. 30, this would only require minor changes In the structure of units 510 
within the scope of the skilled professional. In summary, the search and sort functions of search/sort circuit 502 (one 
search/sort circuit is placed in each neuron circuit 11 of base ZISC chip 10) have been described by reference to the 
circuits of Figs. 29 and 30. Aggregate circuit 51 7 formed by assembling search/sort circuits 502-1 to 502-N is a very fast 
circuit because all the p = 14 bits of all the distance D1 to DN signals are processed in parallel, so that no sequencer is 

55 required. The number of neuron circuits does not impact the overall delay of circuit 518. it just modifies the number of 
inputs of logic gates in dedicated OR sub-unit 12.1 . Finally, aggregate circuit 51 7 is perfectly modular and requires only 
a minimal number of devices. 
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* THE DAISY CHAI N CI RCU IT (600) * 

Fig 31 shows the detailed construction of daisy chain circuit 600 of neuron circuit 11 built around a 1-bit DAISY 
reqister 601 controlled by a signal ST The ST signal is set active during the learning phase in two cases: «")en 'denti- 
f icatlon circuit 400 confirms a neuron circuit should be engaged or at initialization. The ST signal is ^jertlj^^^^^^^ 
from the UNC/FIRE OK* and L signals. The DCI signal is applied to the first input of a 2-way AND gate 602 whose othe 
input receives the T^BSrTsignal. The output of AND gate 602 is connected to the input of register 601. The output o 
register 601 is connected to the first input of a 2-way OR gate 603 whose other input receives the ALL signaL The s gna 
which is outputted from OR gate 603 at the DCO output terminal is the DCO signal. At initialization, the RESET signa 
is set active 1 e Plt5ET-= 0. this "0" value outputted by AND gate 602 is loaded in DAISY register 601 as soon as signa 
ST is set active, so that via OR gate 603 (signal ALL = 0) the DCO signal is set to zero. More ge"«^^;'y;_*^ s^^^^ 
is activated when the data at the output of AND gate 602 needs to be loaded in register 601 . When the ALL signa is 
activated (ALL = 1) the DCO signal is set active (DCO = 1) irrespective the binary value stored in register 60r The ALL 
signal proves usefulness during the test of neural network 1 1(A). As a matter of fact during the test, all "euron 
circuits of the neural network 1 1 (A) without exception behave as if they were engaged. In normal operating conditions 
the ALL signal is set to "0" and the T=ttSET signal is maintained to "1 ". The DCI input terminal of the irst "euron circurt 
11 -1 is connected to VH via an external wire. The ST BESET and ALL signals are applied in parallel to all the daisy 
chain circuits 600 of the neuron circuits 11-1 to 11-N of ZISC chip 10. ■ * ..^ cm th,. 7iqr 

Let us now consider the first neuron circuit of the ZISC chip 10. At initialization, all the registers 601 o the ZISC 
chip 10 hold a zero as a result of the general reset (ffiSfr = 0 and ST = 1) accomplished during 'n't«"fjt;°" 
mentioned above. After the initialization step, the input terminal DCI of the first neuron circuitl 1-1 being tied to VH 
sianal DCI-1 thus applies a one (DCI-1 = 1) to the first input of AND gate 602 and signal T^ESET also applies a one to 
the other input thereof, as a result, a one is applied at the register 601 input. On the other hand, a zero being loaded in 
register 601 the DCO-1 signal which is finally outputted from OR gate 603 (ALL = 0) is equal to 0. Therefore, only the 
DCI and DCO signals of the first neuron circuit 1 1 -1 of neural network 1 1(A) have complementary values, i.e. obey to 
logic relation DCI = DCO because, for all the other daisy chain circuits i.e. of neuron circuits 1 1-2 to 11-N. signa DCi _ 
signal DCO = 0. Neuron circuit 1 1 -1 is thus the ready to learn neuron circuit by construction, as soon as the '"'t'al'zat'on 
step has been completed. When signal ST is set active in the learning mode, the logic one which was applied to the 
input of register 601 is then loaded therein. As a result, signals DCI-1 and DCO-1 are made equal to 1 . the first neuron 
,0 Circuit 1 1 -? has learned and is now engaged. On the other hand, DCI-2 = 1 and DCO-2 = 0 for the 

explained above. This means the following neuron circuit in the chain, i.e. neuron circuit 11-2, is now the ready to learn 
neuron circuit. The reasoning can be extended until neuron circuit 11-N is engaged. Because, for each "euron circuit 
generation of the DCO signal just requires a register and two elementary logic gates at the maximum, the overall delay 
is thus very short and moreover independent of the number of neuron circuits forming neural network 11(A) 
,5 Daisy chain circuit 600 further comprises a control logic circuit 604. which in turn consists of two logic gates: a two- 
way XOR gate 605 and a two-way AND gate 606. XOR gate 605 is driven by the DCI and DCO signals of neuron circuit 
1 1 and generates the RS (Ready to Store) signal for the RAM memory 251 . Basically, the RS signal operates as a write 
enable signal to allow or not the WRITE operation therein. In the recognition phase, the input vector 'S preserted to aH 
committed neuron circuits (selected and engaged), but the input vector components are only pre-charged into the RAM 
40 memory 251 of the ready to learn neuron circuit in prevision of a possible engagement during a subsequent learning 
phase To pre-charge the RAM memory 251 , during the recognition phase has significant advantages, because to store 
the 64 components thereof is really a time consuming process. With the present ZISC chip 1 0 implemerrtaton^ it prac- 
tically just remains to store the input category and to activate signal ST to set register 601 of the ready to 'ea^ neuron 
circuit to terminate the learning phase. Then, the RS signal is set inactive to prevent any further writing in the RAM 
45 memory 251 . On the other hand, AND gate 606 generates the CO signal from the DCO and NS s-gnals. The NS signa 
is internally generated by matching circuit 150 in each neuron circuit as mentioned above (see Fig. 5). The CO signal 
is thus set active for a neuron circuit which is selected and engaged (I.e. committed). Thanks to this combination of 
signals the CO signal is judiciously applied to certain blocks of ZISC chip 10 illustrated in Fig. 4(A). so that only the 
committed neuron circuits are allowed to participate to the recognition phase and to the reduction process in the learning 
50 phase. 
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TABLE III below summarizes the neuron circuit states versus the logic value of DCI and DCO signals. 



TABLE III 



DCI 


DCO 


NC State 


0 


0 


free 


1 


0 


free (ready to learn = first free) 


1 


1 


engaged 


0 


1 


reserved 



TABLE Ml clearly illustrates the two basic states of a neuron circuit: free (including the ready to learn case) and 
15 engaged. 

The DCO signal of the last neuron circuit of a single ZISC chip 10 , i.e. DCO-N, can be used to indicate if the last 
neuron circuit of neural network 1 1 (A) is engaged or not. In other words, it may play the role of a FULL signal. If neuron 
circuit 11 -N is engaged, this means neural network 1 1(A) is full. It is also to be noticed that the critical delay path of the 
neural network is substantially the one of the daisy chain circuit of the ready to learn neuron circuit. The other daisy 

20 chain circuits of the neural network are not impacted and there are no changes in their DCI and DCO signals. 

The daisy chain circuit described for the single ZISC chip 10 of Fig. 4(A) is perfectly adapted to operate In a multi- 
chip environment should several chips be connected in series such as Illustrated in Figs. 1 1 and 12. As more apparent 
from Fig. 11, only the DCI input terminal of the first neuron circuit of the first ZISC chip 10(1) is connected to VH, sb.that 
it is the ready to learn neuron circuit by construction after initialization. On the other hand, the DCO signal of the last 

25 ZISC chip 10(Q) forming elementary module 1 0(A) can be used to indicate if the DAISY register of the last neuron circuit 
of the last chip is loaded with a one or not. If it is loaded with a one, this means all the neuron circuits forming elementary 
neural network 1 1[A] incorporated in elementary module 10[A] are engaged. 

The daisy chain circuit 600 is thus an essential feature for the cascadability of the ZISC chip 1 0, because unlike the 
architecture described in US-A-4974169 mentioned above, the addressing of the ZISC chips 10(1) to 10(Q) is not 

30 required. This means that the conventional decoder and pointer circuits are saved in each chip, which in turn, result in 
a significant circuit density increase. As another consequence, the address bus, with its Inherent limitations as to the 
number of neuron circuits that can be addressed, is also no longer required. As apparent from Figs. 4(A) and 4(B), there 
Is no bus labelled ADDRESS to interconnect the neuron circuits of base ZISC chip 10 and the micro-controller/user 22. 
Consequently, and at least theoretically, an unlimited number of base ZISC chips 10 may be connected in parallel on 

35 the INDATA-BUS. the SEL/CTL-BUS and the GDATA-BUS. only the daisy chain connection being of the serial type. No 
supervisor is necessary for data management during the learning phase to indicate via an address bus, which neuron 
circuit has to learn. According to the present invention, in the learning phase, the role of micro-controller/user 22 is not 
to supervise the operation of the neuron circuits of ZISC chip 1 0. but just to provide them with examples and associated 
categories. The neuron circuits of the ZISC chip 10 practice a sort of self-learning. This is so true that the decision to 

40 learn or not collectively belong to all neuron circuits and not to a supervisor. 

The above described approach of the daisy chain circuit in an application to neural networks can be generalized 
without difficulty to a multi -processors 64 unit comprised of a plurality of base processing units, such as illustrated in 
Fig. 32, by a single base processing unit referenced 65. Each processing unit is connected to a common input data bus 
labelled DATAIN-BUS, via a bi-directional data bus. Each processing unit includes a R/W memory circuit 66. The data 

45 are presented on the DATAIN-BUS to all the processing units in parallel. A daisy chain circuit 67, having the same 
construction as daisy chain circuit 600 of Fig. 31 , i.e. it is basically organized around a DAISY register, is associated to 
each processing unit and generates a RUN signal (equivalent to the CO signal) and a LOAD signal (equivalent to the 
RS signal) that are applied to the appropriate blocks of the processing unit. A serial connection DCI/DCO is established 
between the daisy chain circuits of the multi-processors unit 64. Each daisy chain circuit 67 receives a NEXT signal 

50 (equivalent to the ST signal) generated by a state machine (not shown) to shift the logic value applied to the input of the 
daisy chain circuit 67 by the DCI signal to be stored in the DAISY register of only one determined processing unit. The 
loading of a specific input data presented on the DATAIN-BUS in the R/W memory circuit 66 of such a determined 
processing unit 65 (in fact, the ready to load processing unit) is thus automatically performed by the daisy chain circuit 
whose logic input and output signals DCI and DCO have complementary values. 

55 

* THE INTER-NEURON COMMUNICATION SYSTEM * 

Fig. 33 schematically shows the structure of the interneuron communication system when the ZISC chip 10 archi- 
tecture of Fig. 4(A) is used. The inter-neuron communication system comprises OR circuit 1 2 which generates the global 



39 



BNSDOCID:<EP 0694e55A1> 



EP 0 694 855 A1 



result signals F* DEGOUT* and UNC/FIRE.OK* on the R*-BUS and the global output signal OUT* on the OUT*-BUS. 
These two buses are merged In the on-chip common communication bus COM*-BUS through electrical connections. It 
further comprises the O R-BUS which is directly derived of the OUT*-BUS and essentially transports distance or category 
type of data. 

Now turning to Fig 33. with respect to neuron circuit 1 1 . only identification circuit 400 and Dmin determination circuit 
500 thereof have been schematically represented. As apparent from Fig. 33. all the local result and output signals that 
are locally generated by each of the neuron circuits 1 1 -1 to 1 1 -N, are applied to respective dedicated OR sub-circuits. 
which all together form OR circuit 12. The NOUT local output signal generated by each neuron circuit, which transporte 
the distance/category data, is applied to the dedicated OR sub-circuit 12-1. The OUT* global output signal generated 
by OR sub-circuit 1 2-1 is transported by a bundle of fourteen connecting wires forming the OUT*-BUS. The OUT* signal 
is reinjected via the OR-BUS in each neuron circuit of the ZISC chip 10 as a feedback signal. The F. DEGOUT and 
UNC/FIRE OK local result signals of each neuron circuit are respectively applied to OR dedicated sub-circuits 1 2-2, 1 2- 
3 and 12-4 Global result signals F*. DEGOUT* and UNC/FIRE.OK* generated by said OR dedicated sub-circuits 12-2 
to 12-4 are then available on their respective global result wires which all together form the R*-BUS. OR sub-circuit 12- 
1 is constructed by a battery of fourteen N-way OR gates. Each OR sub-circuit 12-2 to 12-4 is constructed from a N- 
way OR gate Alternatively. OR sub-circuits 12.1 to 12.4 may be constructed by an equivalent arborescent structure as 
standard However, it must be noted that an OR dotting could have been be used instead of the OR function performed 
in hardware by OR circuit 12 if permitted by the design system and the technology. The global result and output data 
are preferably stored in register 184.1 for further processing. The ID* signal is reconstructed in AND gate 184.2 from 
the F* and UNC/Fmt.'DK'signals. The ID* signal is also stored in register 184.1 but may be outputted outside the ZISC 
chip 10 as shown in Fig. 33 for connection to a LED to provide an immediate visual information to the user as to the 
global response of neural network 1 1(A). Fig. 33 shows the implementation of the inter-neuron communication systerri 
of the present invention organized around the COM*-BUS for the ZISC chip 10 of Fig. 4(A) incorporating neural network 
1 1(A) for stand-alone operation. However, this architecture can be improved to allow the assembly of a plurality of ZISC 
chips 10 to produce neural networks of bigger size. 

Let us now consider the case the base ZISC chip 10 whose architecture has been adapted to operate in a multi- 
chip environment as described above by reference to Fig. 4(B). Now tuming to Fig. 34, for each base ZISC chip 1 0. the 
said connecting wires forming the R*-BUS and the OUT*-BUS now connect the outputs of said OR sub-circuits 12-1 to 
12-4 to respective driver circuits (e.g. sub-block 19.1 includes the driver circuits of the DR* type that are connected to 
each wire of the OUT*-BUS) These driver circuits are adapted to provide an additional OR dotting function on the off- 
chip common communication bus labelled COM**-BUS which is thus shared by all the ZISC chips 10 connected thereon. 
Global result signals F**. DEGOUT** and UNC/FIRE.OK** and global output signal OUT** are then available on their 
respective global wires of the COM**-BUS. All or part of these global result and output signals are reinjected in each 
ZISC chips 10 of the neural network via adequate receiver circuits (RR) still in block 19 placed in each ZISC chip 10 as 
apparent from Fig 34 However, it should be noted that a not-dotted OR function, for instance using an OR circuit such 
as 12 could be used in lieu of the above described OR dotting function. Unlike the architecture of Fig. 33 which includes 
only one OR stage, the architecture of Fig. 34 includes two OR stages. However, only one OR stage could be envisioned 
as well, as described above by reference to the architecture of the ZISC chip 1 0" of Fig. 1 0. When the distinction between 
the COM*-BUS and the COM**-BUS is irrelevant, any common communication bus will be referred to as the COM-BUS. 
Multiplexer 21 shown in Fig. 4(B) which allows the selection of either the OUT* signal or the OUT** signal to be reinjected 
as the OR signal in each neuron circuit 1 1 is thus a convenient means to have the base ZISC chip 10 to operate either 
in a single chip or multi-chip environment. 

Fig 35 shows details of a typical implementation of the driver sub-block 19.1 of Fig. 34 in the construction of the 
elementary module 10[A1 of Fig. 1 1 . Now turning to Fig. 35. there is shown the implementation for the first bit of eight 
base ZISC chips 10 respectively referenced 10(1) to 10(8) which all together form the elementary module 10[A]. The 
latter incorporates elementary neural network 1 1[A]. As apparent from Fig. 35. a hard-wired AND dotting function has 
been implemented in reality on the GDATA-BUS. instead of an OR dotting function for practical reasons given just below. 
An OR dotting function would have required PFETs which are slow devices. Therefore, NFETs which are known to be 
fast switching devices are used instead. In the present case, each driver circuit DR* thus simply consists of a NFET 
device connected in an open drain fashion as standard. According to a bipolar technology, NPN transistors connected 
in open collector fashion would constitute an alternate solution. All the first bit signals for the eight chips, i.e. bit signals 
OUT*1(1) to OUT*1(8) are applied to the gates of their respective NFET devices whose drain is dotted on a common 
wire W1 loaded by pull-up resistor RL1 connected to the first potential VH. The source of each NFET device is connected 
to the second potential GND. As a final result, the desired AND function between said bit signals is performed on the 
corresponding wires of the GDATA-BUS to produce bit signal OUT**1. The OUT** signal that is now obtained through 
an AND function has an opposite polarity to the OUT** signal discussed above that was obtained through an OR function 
(e g with the dotting implementation of Fig. 34). It is interesting to notice that physically, the dram of each NFET device 
is not directly connected to the input of the receiver circuit RR inside the chip. For each Zl SC chip 1 0. the drain connection 
is outputted from the chip to be connected to a wire W1 . This common wire W1 shared by all the NFET devices transports 
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the OUT**1 bit signal to the GDATA-BUS. The 0UT**1 bit signal is returned back to each 2ISC chip 10 via wire Z1 and 
a receiver circuit RR. A strap STR1 interconnects wires W1 and Z1. The construction shown in Fig. 35 may be easily 
generalized to the other remaining bits, so that the 14 bit OUT** signal is totally made available on the GDATA-BUS. As 
illustrated in Fig. 35, receiver circuit RR is a inverting buffer. It may be of any conventional type. It is required to regenerate 

5 the OR1 bit signal for each ZISC chip 1 0 with the adequate polarity, because the driver circuit DR* (which consists of a 
NFET device) operates as an inverter. It may be worthwhile to apply a control signal labelled PT (PT stands for Pass- 
Through) to receiver circuit RR so that when signal PT is set active, the OR1 signal is forced to "0". As mentioned above, 
by reference to Figs. 29 and 30. when the OR signal is set to zero, search/sort circuit 502 is passed through (i.e. the 
exclude function mentioned above is inhibited). The resulting elementary ZISC module 10[A] is thus comprised of a 

w number Q (in the present case, Q = 8) of base ZISC chips 1 0 and does not require any external component. This results 
of the particular technique of dotting including the strap technique that is described above. Should the connection be 
made directly between the output of the driver circuit DR* and the input of the receiver circuit RR, such flexibility would 
have been limited. The above construction may be further extended to each of the global result signals F**, DEGOUT** 
and UNC/FIRE.OK**. 

15 Fig. 36 is comprised of parts (A). (B) and (C). Parts (A) and (B) show the correspondence between: (A) the OUT* 

vs the NOUT signals and (B) the OR vs the OUT* signals according to the implementation of Fig. 33. Part (C) shows 

the OR/OUT** vs the OUT* signals according to the implementation of Fig. 35. 

The search time in the determination of Dmin and the sort time that is required to sort the remaining distances are 

verydependentof the delay of OR sub-circuits 12.1 to 12.4(Fig. 34) and of driver circuits (and thus of the load resistors). 
20 To minimize these delays, the solution described in Fig. 37 is recommended. This solution directly derives from the 

construction shown in Fig. 35, but takes profit of the optional implementation of the main Dmin determination circuit 20 

in each base ZISC chip 10. 

Now turning to Fig. 37, there is shown the logical implementation of a variant of the elementary module 10[A]t still ■ ^ 

formed by the assembly of eight base ZISC chips referenced 10(1) to 10(8) but when each of them is provided with its 
25 respective main Dmin determination circuit 20. In ZISC chip 10(1), aggregate circuit 517 (see Fig. 30) generates the 
NOUT1-1 to NOUTp-N signals that are applied to the dedicated OR sub-circuit 12.1. Signals supplied by OR sub-circuit 

12.1 are labelled OUT*1 to OUT*p and are applied to main Dmin determination circuit 20. With the present implemen- 
tation of the base ZISC chip 10, N = 36 and p = 14. The OUT* signal comprised of bit signals OUT*1 to OUT*p outputted 
by OR sub-circuit 12.1 is now applied to main Dmin determination circuit 20 as an input signal. The bundle of wires 

30 transporting the OUT*1 to OUT*p bit signals is also applied to aggregate circuit 51 7 as the OR-BUS (for circuit 517, the 
OR signal is the same as the OUT* signal). In turn, the signals that are outputted from main Dmin determination circuit 
20 are applied to their respective NFET devices forming the driver circuits DR* of sub-block 1 9. 1 . The bit signals OUT**1 
to OUT**p that are outputted from the ZISC chip 10(1) on the drains of the NFET devices, are available on respective 
wires W1 to Wp connected to VH via respective load resistors RL1 to RLp. Once the OUT** bit signals have been ANDed 

35 for the 8 chips and for each of the p bits, the corresponding bit signals 0UT**1 to OUT**p of signal OUT** are available 
in the GDATA-BUS. The OUT** signal is applied to main Dmin determination circuit 20 as the OR*-BUS (for circuit 20. 
the OR* signal is the same as the OUT** signal after complementation). The path which conveys the OUT**1 to OUT**p 
bit signals to main Dmin determination circuit 20 includes straps STR1 to STRp. wires Z1 to Zp and respective receiver 
circuits RR. Any main Dmin determination circuit 20 has the same construction as search/sort circuit 502. The eight 

40 ZISC chips forming the elementary module referenced 10[A] in Fig. 37, are thus working in parallel to determine first 
the minimum distance value among the distances calculated by the neuron circuits in each ZISC chip 10, then, the 
minimum minimorum distance among said minimum distances calculated at the elementary module level. With the 
implementation illustrated in Fig. 37, the search and the sort processes are very efficiently and fastly performed. The 
limitation of the chip cascadability to a number of eight in the described example, is due to the maximum number of AND 

45 dotting functions accepted by the technology. Derived from the teachings of Fig. 35, the circuit of Fig. 37 illustrates the 
practical way to cascade eight base ZISC chips 10, each being provided with a main Dmin determination circuit 20 to 
form an improved variant of the elementary module 1 0[A] of Fig. 1 1 . In turn, such elementary modules may be assembled 
to construct complex modules which incorporate corresponding complex neural networks. 

Such a highly desirable result may be attained through the logical architecture of Fig. 38. In this case there is no 

50 practical limitation to the elementary module cascadability. Now turning to Fig. 38. there is described the construction 
of a complex module labelled 10{A} to illustrate how a number R of such elementary modules 10[A] can be assembled. 
Each elementary module 10[A] is comprised of eight ZISC chips referenced 10(1) to 10(8). For sake of simplicity, let us 
just consider the first bit signal 0UT**1 which is generated by each elementary module. The OUT* *1 bit signals generated 
by the R elementary modules are inputted in a R-way AND gate 68.1 to generate the OUT***1 bit signal. The third 

55 asterisk indicates that a third logic function (in this case an AND) has been performed. In turn, the 0UT***1 bit signal is 
applied to each of the eight ZISC chips of each elementary module via an inverting receiver circuit RR (not shown) to 
produce the OR1 bit signal as explained above. AND gate 68.1 ensures consistency with the implementation shown by 
reference to Fig. 35. The reasoning may be applied to the other bit signals that are processed in respective AND gates 

68.2 to 68.p to generate the OUT***2 to OUT***p which in turn, feeds the OR2 to ORp bit signals respectively. The AND 
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circuits 68. 1 to 68.p may be functionally built by OR. NOR or NAND gates and the like as well, as far as the desired AND 
logic function is performed. 

The description made by reference to Fig. 38. clearly demonstrates that only a few number of devices are required 
to implement the inter-neuron communication system including up to three logic functions. The COM"*-BUS which is 
5 the essential component of the GDATA-BUS, can theoretically connect together an unlimited number of base ZISC chips 
10 and thus of neuron circuits. Each neuron circuit sends its local result and output signals on the NR-BUS and the 
NOUT-BUS respectively at the same time for simultaneous processing in OR circuit 12. Next, the NR* and OUT* that 
are outputted by OR circuit 12 are then AND dotted on the GOM"-BUS. which again allows a fast parallel processing 
of these signals. Finally, these two successive OR and AND stages result in a low overall processing delay because of 
10 the fully parallel architecture of the base ZISC chip 10. The GOM"-BUS is also an essential element to full chip cas- 
cadability discussed above by reference to the daisy chain circuit 600. As mentioned above, the OR-BUS is applied to 
each neuron circuit of either the single ZISC chip 10 of Fig. 4(A) or to each of the base ZISC chips 10 of Fig. 4(B) that 
are assembled to form an elementary or a complex module. In addition, several hardware solutions are available to the 
user to realize any global logic function on the COM"-BUS, such as, but not limited to. OR-dotting and/or AND-dotting 
15 functions Note also that with the architecture of ZISC chip 1 0" of Fig. 1 0. only one OR stage is required. This OR stage 
is preferably performed by OR dotting via driver circuits labelled DR*. In this case, the inter-neuron communication 
system only includes block 19 (where the AND dotting is physically performed instead the OR dotting for the practical 
reasons explained above), the COM**-BUS and the OR-BUS which directly derived therefrom. 

The above description relates to an inter-neuron communication system and therefore appears to be limited to a 
specific application to neural networks constructed either by a base ZISC chip 10 or by combining the desired number 
thereof However this approach may be generalized, in particular to a multi-processor unit comprised of a plurality of 
base processing units. In fact, this approach is valid everywhere it is possible to send local result data generated by all 
or at least a determined number of processing units at the same time on a common communication bus and to have 
said processing units to take part to the determination of a global result. Such determination typically consists in per- 
forming a desired logic function of the local result data to produce a global result data available on the said common 
communication bus. The global result data may be directly accessed on this bus or stored in a register. Finally, a compare 
circuit placed in each processing unit then compares the local data to the said global data which is transported therein 
via a feedback bus. In turn, the result of the comparison which is a new local result data can be subsequently used to 
initiate various actions, such as to inactivate certain resources or blocks of a determined processing unit from the group 
of processing units or to compute still another global result data as well. Each local result signal transported on a local 
line or bus is thus the result of a local evaluation whose status is dependent of the processing unit in consideration. On 
the contrary, any global result signal transported on a global line or bus is the result of a global evaluation performed at 
the level of all base processing units. ^ • *u 

In the case of the neural network 1 1(A) of Fig. 4(A). neuron circuits 11-1 to 1 1-N are interconnected via the OR 
35 Circuit 1 2 the on-chip common communication bus COM*-BUS and the OR-BUS as a feedback bus. An essential problem 
mentioned above is to find if all the committed neuron circuits that have fired have the same category or if they belong 
to at least two different categories. To that end. the local category C (held in the category register 450) and the global 
category C* (via the OR-BUS) are compared in identification circuit 400. In this example, the neuron circuit 1 1 plays the 
same role as the base processing unit mentioned above. The local category C held in the category register 450 is a 
local data and the global category C* is the global data obtained by ORing all the local categories in OR sub-circuit 12.1 . 
The global category 0* is transported to each neuron circuit via the OR-BUS which thus plays the role of the feedback 
bus for being compared with the local category C held therein. The result of the comparison is signal UNC/FIRE.OK 
which is a local result. In turn, all local result signals UNC/FIRE.OK are ORed in circuit 1 2 to generate the UNC/Fl RE.OK* 
signal which is a global result. For instance, this comparison is performed during the recognition phase according to box 
45 41 Of Fig. 14(B). Finally, the COM**-BUS is composed only by a plurality of global result lines transporting global result 
signals (F* DEGOUT* and UNC/FIRE.OK*) and of global output lines transporting the global output signal (OUT**). 

Let us assume now that each base processing unit has one output bus to transport a local result signal and one 
input or feedback bus transporting the global result. A number of base processing units can be connected that way by 
the use of well established connection rules. Fig. 39 gives an example of this configuration in the case a dotted AND 
50 function is used, so that the implementation is very simple. Now turning to Fig. 39. there is shown an elementary process- 
ing unit 69 comprised of a number N of base processing units referenced 70-1 to 70-N. The strap function mentioned 
above that is required between the global result bus W and the feed-back bus Z is performed by a strap bus STR. A 
load resistor RL is adequately connected to each line of the global result bus W as schematically illustrated in Fig. 39. 
Note that the strap function may be replaced by a driver circuit. 

The above approach illustrated by reference to Fig. 39, may be further complicated for maximal generalization. In 
case a high number of base processing units have to be interconnected, this number may have a limit imposed by the 
technology Beyond this limit, an intermediate merge block would be required to combine two or more of said global 
result signals, now considered as intermediate result signals to produce a new global result signal on a global result 
bus Finally this global result bus is connected as a feed-back bus to each of said base processing units via the strap 
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bus. Due to this structure, no supervisor is needed for data management, only the strap connection between the global 
result line and the feedback line is in fact required. The reasoning may be even further generalized, new global result 
signals, in turn, becoming intermediate result signals that can be combined in another intermediate circuit to produce a 
final global result signal on a final global result bus. 
5 Fig. 40 illustrates this generalization in some extent. Two elementary processing units such as described in con- 

junction with Fig. 39. referenced 69[1] and 69[2] are assembled to produce a complex processing unit 71 . The respective 
global buses labelled W1 and W2 are applied to AND gate 72 whose output, via strap bus STR, forms the feedback bus 
Z that feeds each base processing unit 70-1 to 70-N of each of elementary processing units 69[1] and 69[2]. 

10 * ANCILLARY CIRCUITS * 

A convenient way of loading an influence field value that is always less than MaxlF in the AIF register 352 is to use 
the simplified version of the search/sort circuit 502 described by reference to Fig. 29. Now turning to Fig. 41 , there is 
described circuit 73 which illustrates the detailed construction of this additional circuit referenced 502' in a base ZISC 

75 chip 10. The role of circuit 502* is to clamp the influence field at the MaxlF value. Clamp circuit 502' is built with the same 
elementary base units 510 of Fig. 28 that are cascaded in series the same way. The main differences with regard to the 
search/sort circuit 502 of Fig. 29 are not in terms of construction but rather in terms of connections. The input signal 
that is applied to clamp circuit 502' is MaxlF instead of the calculated distance D. When clamp circuit 502' is needs to 
be selected (not inhibited), its EXCLIN input is connected to the ground potential GND. On the other hand, its EXCLOUT 

20 signal is not outputted. With the described implementation of Fig. 41. the OR gates of dedicated OR subcircuit 12.1 of 
Fig. 29 must be provided with an additional input, they are now labelled 12'.1 to 12'.4 (they have N+1 inputs instead of 
N inputs). Only one clamp circuit 502' is implemented in a base ZISC chip 10 and it participates in the same way to. the 
determination of the minimum distance Dmin. The role of this clamp circuit 502' is thus to set the AIF register 352!6f a 
newly engaged neuron circuit to the MaxlF value during the learning phase. For instance, this can occur when none 

25 neuron circuit of the base neural network 1 1 (A) having learned, the MaxlF value is loaded in the ready to learn neuron 
circuit. 

THE BASE ZISC CHIP MAJOR FEATURES 

30 The base ZISC chip 10 of Fig. 4 (B) is adapted to learn and recognize input vectors up to 64 components (entries). 
Because of its parallel architecture, the ZISC chip 10 simultaneously computes the distance between the input vector 
and all the prototype vectors previously stored therein (neuron circuits having "learned") . The distance calculation method 
is based on a norm that is user selectable. Note that this norm may be different from one neuron circuit to another one. 
In the above description, the two available distance computation methods are based upon the LI and Lsup norms, 

35 however, other calculation methods may be envisioned as well. The limitation of N (N = 36) neuron circuits in the Base 
ZISC chip 10 is not a problem because of the full cascadability of the chips. As a consequence, an unlimited number of 
prototype vectors can be stored in the neural network. During the learning phase, each neuron circuit automatically 
adjusts its influence field in order to optimally map the feature space. This adjustment occurs simultaneously in all neuron 
circuits without requiring any external action, such as a supervision by a micro-controller As a result, the learning process 

40 is performed internally and is fully automatized. On the other hand, the learning time is independent of the number of 
neuron circuits forming the neural network. In addition, to further save time, the pre-charge step allows to have the input 
vector components stored during the recognition phase, so that a new presentation of the input vector is not necessary 
for learning. The ZISC chip 10 is provided with a high number of categories (16384) and further includes a context 
feature which in particular allows to divide the neural network in a number of sub-sets. It is also provided with an efficient 

45 search and sort scheme for fast minimum distance determination and efficient sorting of the remaining distances. The 
base ZISC chip 10 can operate with both SAVE and RESTORE modes for increased efficiency. Finally, the number of 
neuron circuits forming any neural network is unlimited and transparent for the user. 

MAIN APPLICATIONS OF THE ZISC CHIP 

50 

The base ZISC chips 10 of the present invention.are based on a highly innovative neuron circuit architecture. As a 
result, that allows the construction of elementary and complex modules that incorporates neural networks of any desired 
size, rating the disclosed innovative neuron circuit structures according to the present invention are adapted to meet a 
number of applications in various fields of the technology. The ZISC chip provides an innovative and low cost solution 
55 in many domains such as : 

Image recognition (factory automation, ... ) 

Image compression (data storage, ... ) 

Video compression (video storage, video conference. ... ) 

Character recognition 
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Voice processing and recognition 
Signal recognition (radar, sonar and the 
Data analysis 
Robotics 

Equipment monitoring 
Signal processing 
etc. 
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NR-BUS the bus transporting the NR signal 
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OUT* global output signal (one ORing) 
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p width of various buses of the ZISC chip 
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Claims 

1 . In a neural network comprised of a plurality of N neuron circuits (11 -1 ..... 1 1 -i. .... 11 -N). each neuron circuit having 

calculated the distance (D1 ..... Di DN) coded on p bits (Di = d1 i ... dki ... dpi) between an input vector (A) and 

the prototype vector (B) stored in the weight memory thereof, the aggregate circuit (51 7) including: 
acorresponding plurality of search/sort circuits (502-1. ...,502-i. ... . 502-N) each being placed in a respective neuron 
circuit; wherein each search/sort circuit (502-i) is comprised of p elementary base units (51 0-1 i to 51 0-pi) connected 
in series and disposed along a column; wherein each elementary base unit (510-ki) receives three input signals: a 
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distance bit signal (dki), an exclude input signal (EXCLINki) and a feedback signal (ORk) and generates two output 
signals: a local (or neuron) output signal (NOUTk-i) and an exclude output signal (EXCLOUTki); said distance bit 
signals being arranged along a line for each base unit of the same bit rank, so that the base units of said aggregate 
circuit are organized In a matrix; wherein the exclude output signal of said base unit is applied as the exclude input 
signal of the next base unit in the column; wherein the local output signal (NOUTk-i) is equal to the complemented 
value of the distance bit signal (dki) if the exclude input signal is Inactive (EXCLINki = 0) and to the true value thereof 
if active; wherein the exclude output signal is active (EXCLOUTki = 1 ) if the feedback signal (ORk) is complementary 
to the distance bit signal; and, wherein the feedback signal (ORk) corresponds to the signal obtained by performing 
an OR function between all the local output signals (NOUTk-1 to NOUTk-N) generated by the base units of line k 
for all the columns of the matrix. 

2. The aggregate circuit of claim 1 further Including: 

first logic circuit means (520) comprised for each search/sort circuit (502-i) of inverting means (524-i) con- 
nected to the base unit (510-pi) processing the last bit (dpi) to produce a signal (XOR-i) equal to the complement 
value of the exclude output signal (EXCLOUTpi) generated therein; and. 

second logic circuit means (519) comprised for each search/sort circuit (502-i) of a two-way OR gate (522- 
I) connected In series with a latch (523-i) whose output is connected to the base unit (510-11) processing the first 
distance bit signal (d1 1) as its exclude input signal (EXCLIN-i) and to the first input terminal of the said two-way OR 
gate whose other input receives the signal generated by the corresponding inverting means (XOR-i). 

3. The aggregate circuit of claim 2 further including: 

an two-way OR gate (521-1) for each search/sort circuit (502-i) inserted between the latch and the said base unit 
that is driven on one input by the signal outputted by the latch and on the other input by a selection signal (-SELECT-i). 

4. The aggregate circuit of any above claim wherein said elementary base unit (510-ki) consists of: 

a two-way NOR gate (513) driven by said distance bit signal (dki) and said exclude input signal (EXCLINki) 
to generate said local output signal (NOUTk-i); 

a two-way AND gate (51 4) driven by said distance bit signal (dki) and said feed-back signal (ORk) to generate 
an Intermediate signal; and 

a two-way OR gate (515) driven by said intermediate signal and said exclude Input signal (EXCLINki) to 
generate said exclude output signal (EXCLOUTki). 
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